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ABSTRACT 

These appendices accompany a study that was conducted 
to determine whether the state and federal performance standards for 
Job Training Partnership Act (JTPA) programs have influenced Service 
Delivery Area (SDA) and service provider decisions about program 
design and implementation practices in wa/s that have affected the 
clients served, the types of services provided, and the costs of the 
services. The first appendix summarizes the study *s four hypotheses. 
Case study findings and conclusions are listed in the second 
appendix. Averages of the independent variables used in the 
quantitative analysis are presented next. The fourth appendix is a 
chart that cross-references SDA goals, strategies for achieving 
performance goals, and performance standards in the 1986 program 
year. The final appendix deals with the study's data collection 
instruments. It covers the distribution of responses to the 
questionnaires and presents an outline of topics for on-site 
discussions. (MN) 
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Appendix A. 
Summary of Hypotheses 



Appendix A 
SUMMARY DF HYPOTHESES 



Program-design elements affect clients, services, and costs. 

A. Enrollment process ami criteria affect characteristics of clients. 

1. When outreach and intake are conducted by service providers, the providers' own 
preferences and the incentives created by contractual arrangements are likely to have a 
strong influence on whom is enrolled. 

2. SDAs that establish priority groups for service are more likely to serve the hard to 
serve. 

3. SDAs that have established tests and other criteria to exclude applicants are more likelv 
to serve the more job ready. 

4. SDAs that take all eligible applicants serve more disadvantaged. 

5. SDAs that establish restrictive enrollment criteria have recruitment and expenditure 
difficulty. 

B. The mix of program services affects clients, services, and costs. 

1. SDAs that link basic skills training with employment-oriented programs will be more likely 
to serve less .job-ready Individuals. 

2. SDAs that emphasize employer needs are more likely to enroll the less disadvantaged. 

3. SDAs that have established YEC systems are more likely to enroll less job-ready youth. 

4. SDAs that arrange joint funding of their programs with other programs and agencies will 
have lower reported costs. 

C. Service-provider arrangements affect clients, services, and costs. 

1. The type of service providers may have a strong influence on whom is enrolled and the type 
of program activity received, particularly in SDAs where they conduct intake activities. 

2. Because of differences in orientation, the type of service provider may also affect the 
length of services provided and. therefore, the cost of services. 

3. The use of performance-based contracts may lead service providers to select the more 
job-ready individuals because they will need to expend fewer resources and have a higher 
probability of receiving full payment by placing such individuals. 

4. The use of performance-based contracts may reduce the length of stay in the program 
because the provider has an incentive to place individuals as soon as they are employable. 

5. SDAs that vary the terms of performance-based contracts may serve more hard-to-serve 
individuals or provide more intensive program services. 

6. Full payment for placements in jobs that last a specified length of time may result in 
longer and more expensive services. 

7. Higher wage-rate goals may increase enrollment of more employable clients but may also 
increase incentives to provide longer training. 

8. Reserving a large fraction of payment for placement and paying an additional amount for 
meeting a placement goal are expected to intensify the incentives to serve the more 
job ready and to provide the minimum training to achieve the outcome. 

9. Performance-based contracts that specify entered-empl oyment-rate goals and pay extra for 
n.ceting goals may lead providers to emphasize quick placements. 

10. Incorporating goals for service to specific client groups in contracts may increase 
service to the hard to serve. 
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Appendix A (Continued) 



Federal performance standards affect clients, services, and costs. 

A. Federal performance standards may affect the enrollment process. 

1. Federal performance-standards policies may have the Intended effect of not influencing the 
clients enrolled in JTPA programs. 

2. SDAs may set up procedures to enroll the easier to serve because the adjustment models may 
not be adequate to hold SDAs harmless for enrolling the hard to serve. 

3. Frequent changes in the DDL models may reduce confidence in the models and thereby lead 
SDAs to establish procedures to enroll the less hard to serve. 

4. The specific mix of standards may lead SDAs to establish client-selection criteria to 
enroll those for whom the performance outcomes are more appropriate. 

5. The level of standards may be set so high that SDAs cannot meet their standards without 
establishing criteria to select clients who are most job ready on the basis of 
characteristics not Included in the model . 

B. Federal performance-standards policies may Influence the mix of program services offered. 

1. Performance standards may have the Intended effect of guiding SDAs to choose adult 
programs that are enployment oriented, that train clients for high-quality jobs and that 
are cost-effective and to choose youth programs that provide an appropriate combination of 
employment-related programs and other-outcome-oriented programs that are cost-effective. 

2. Termination-based performance standards may induce SDAs to provide program: with inmediate 
employment as the goal, such as job-search assistance, rather than to provide more 
Intensive services. 

3. The posltive-termination-rate standard may lead PICs to establish YEC systems that require 
minimal effort to achieve. 

4. The cost-per-entered-employment standard may be set so low that more intensive 
Interventions are not feasible, including the provision of basic skills training and the 
linking of several program services. 

C. Performance standards may affect service-provider arrangements. 

1. Performance standards may have the Intended effect of leading SDAs to choose 
cost-effective service providers and to use with cost-minimizing contractual arrangements. 

2. Performance standards may lead SDAs to use performance-based contracts to reduce the risk 
of not meeting cost standards. 

3. Performance standards may lead SDAs to set terms of performance-based contracts to 
emphasize placement outcomes with low payments to meet the cost standards. 

4. Performance standards may result in SDAs being unwilling to vary the terms of the 
pei'formance-based contracts. 

5. Performance standards may have made SDAs more conservative and risk averse in making 
service-provider decisions and thus less likely to use CBDs. 

State performance-standards policies affect clients, services, and costs. 
A. Additi onal standards may affect the slements of program design. 

1. Equity-of-service standards may increase service to more disadvantaged groups. 

2. Equity-of-sorvice standards may increase the use of CBOs as service providers because of 
their cofmjitment to serving more disadvantaged individuals. 

3. Job-retention standards may create incentives to enroll more job-ready applicants but may 
increase length of stay because SDAs may need to provide more intensive services. 
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Appendix A (Continued) 

B. Procedures for adjusting standards may affect the elements of program design. 

1 Stat(5s that do not use adjustment model jj may increase the incentives of SDAs to serve the 
more job-ready candidates. 

2. Statijs that make adjustments beyond the model may increase the incentives of SDAs to serve 
the harder to serve or to provide longer term program services. 

^' J^^V'M^A^^ adjustments based on State or regional data may mors adequately adjust for 
their SDAs circumstances. ^ ^ 

C. The State's 6% policy m^y affect elements of program design. 

1. State policies that link 6% awards to serving specific client groups will increase service 
to those groups. 

2. Policies gwing greater weight to specific standards will accentuate whatever design 
incentives are associated with those standards. 

3. Policies that give more incentive payments for the extont to which standards are exceeded 
may accentuate the effects of performance standards. 

IV. Other factors may influence program design. 

A. Other Federal policies may affect program design. 

1. The administrative cost limit creates a strong incentive for SDAs to use performance-based 
contracts becau-.e the total costs of such contracts are counted as training costs. 

2. The limit on support costs and work experience affects the types of services offered 
limiting work-experience programs and support services offered to participants. 

3. The support costs limitation may increase the use of performance-based contracts. 

4. JTPA reporting requirements for specific cl .ent characteristic may lead SDAs to increase 
service to those groups. 

5. The costs of Federal reporting requirements may inpose a record-keeping burden on SDAs 
increasing the costs of the program. 

6. Postprogram data collection requirements may impose additional burden and raise costs 
further. 

B. PICs may affect elements of program desigr.. 

1. The extent of influence of the PICs depends on their involvement in JTPA programs. 

2. PICs with more orientation toward serving local employers' needs conpared with serving the 
needs of the disadvantaged may result in programs serving the more job ready and orovidinq 
services such as OJT. - p » 

3. PICs may also accentuate the effects of performance standards if their members see meeting 
those standards and receiving incentive awards as a very importan', goal. 

4. PICs that enphasize performance relative to past years' performance cr relative to the 
national standards instead of relative to the adjusted standard may lead SDAs to serve the 
more Job ready and provide less-intensive, less-expensive services. 

C. Local elected officials may affect elements of program design. 

1. The extent of the influence of the local elected officials depends on the extent of their 
Involvement i.n the JTPA program. 

2. The direction of the influence of the local elected officials depends on their orientation 
toward the needs of employers eid participants. 
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Appendix B. 
List of Case Study Service J . oviders 
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state SOA 



Appendix B 
LIST OF CASE Sm SERVICE PROWDEPS 



Name of Provider 



Type of Provider 



Types of 

Training Provided Type of Contract Target Population 



CA fem/Inyo/Mono Mexican American Opportunity camuiity-based organization woric experience or cost r^ioixirsarent 
Foundation 



CA Kem/Inyo/Mono fern H.S. District 



public edbcational 
institution 



CA Kem/Inyo/Mono Ehployers' Training Resource SiW 



try out erploymant, 
remedial basic edica- 
tion, occt^sational 
classroan training 

remedial basic edica- 
ticn, occupational 
classroom training, 
wrk experience or try 
out erplo^frait, YEC 
cunriculim 



on-the-job training, N/A 
job-search assistance 



cost reirrbursQTEnt 



adults & yoith; 1 pro- 
ject - h.s. dropouts; 
1 project - limited 
English 



adults & youth; 
2 projects dropout 
youth, 1 project in- 
school youth, 1 pro- 
ject disabled and hard 
to serve, 1 project 
limited English 

adult & youth 



CA Los Angeles County Industry Cdnmunity Interface 



CA Los Angeles County th3>'ez Institute 



coriiuiity-based organizatiai work experience (with 

classroom training in 
basic skills) 

jcrnunity-based organization occupational classroan 

training, sore on-the- 



CA Los Angeles Oxjnty Ctmpjn^'ty Bi-lingual Heme 
Health Aid Prpgran 

CA Los Angeles County Vfest San Gabriel Prime otter public agency 

Agent (subcontracts cut 
the training) 

CA Orange County Orange County (not the SDA) other public agency 



job training 

Lontiffiity-basad organization occupational classroom 

training 



occupational classroom 
training, on-the-job 
training 

intake, job-search 
assistance directly, 
referral to other 
programs 



fixed unit price with offenders, welfare 
less than 50% holdback 
for placensnt 

fixed unit price with general (Hispanic) 
less than 50% holdback 
for plac6TBit 

fixed unit price with welfare, Hispanic 
less than 50% holcback 
for placerent 

fixed mit price with general, residents of 
less than 50% holcback area, covered by 
for placement agrearent 



cost reifrbursement 



general, residents of 
area, covered by 
agrearent 



ERIC 



.13 



state SDA 



Name of Provider 



Type of Provider 



Training Provided Type of Contract Target Populat'lon 



' CA Orange County 

; CA Orange County 

CA Soncna Canty 

CA Sonora County 

CA Soncma Canty 

HD Baltimore City 

HD Baltimore Pity 

HD Baltimore City 

HD Baltimore Cbunty 

; HD Baltimore County 

; HD Baltimore County 

* HD Montgomsry 



SER/Jobs for Progress, Inc. canrunity-based organization 



North Orange County Regional ptilic edbcational 
Occtpational P-ograni institution 



Scncnu County Office of Ed. public e Vcational 

institution 



Goodwill Industries 



California HuiBn Dev. Corp. 
Center for Ehployrent and 
Training (CET) 



Urban League 



Harebuildsrs Assoc. of HD 



ITT Career Training Center 



Eastside Ocxupational Training 
Center 



cannnity-based organization 



private nonprofit 
organization 



private for-profit 
organization 



public educational institu- 
tion 



Catonsville Occupational pdalic edcational institu- 
Traip-' Jenter tion 



Catonsville Camunity 
Career Center 



Suburban Md. Building 
Industry Association 



pdblic edbcational institu- 
tion 



private nonprofit organiza- 
tion 



classrocm training, fixed unit price with gf.eral, Hispanic 
occupational classrocm less than 50% holcback 
training for placement 

classrocm training, fixed unit price with general 
occupational classrocm less than 50% holdback 
training for placement 

worf> experience or try fixed unit price with in-school (iojth at 
out erplojmsnt ncre than 50% holcback risk) 

for placeient 



private nonprofit 
organization 



caminity-based organization 



on-the-job training 



occupational class- 
room training 



fixed unit price with adults & youth 
morp than holcback 
for placement 

fixed unit price with adults & youth 
more than 50% holcback 
for placement 



job-search assistance, cost reirrbursement general 
intake and assessment 

occupational classroom fixed unit price with adults & youth. AR)C 
training less than 50% holcback 

for placarent 

occufBtional classroom f iAed unit price with adults & youth. AFDC 
training less than 50% holcback 

for placement 

occupational classroom fixed unit price with adults & youth, AFDC 
training less than 50% holcback 

for placerant 

occupational classrocm fixed unit price with adults & youth. AFDC 
training less than 50% holcback 

for placeront 

remedial basic edjca- fixed unit price with dropout youth 
tion, YEC curriculun less than 50% holcfcack 
for placarent 

occupational class- fixed unit price with adults and youth 
room training less than 50% holcback 

for placement 



state SOA 



Narre of Provider 



Type of Provider 



Training Provided 



Type of Contract 



Target Population 



HD Montgomery 
MD Montgomery 

HD Prince George's 

W) Prince George's 

H) Prince George's 
Boston 



M/V Boston 



W< Boston 



Hontgonery Cdmty PIC 
Tenple School 



Prince George's Camunity 
College 



SOA 

private edxational 
institutiai 



public edxational 
institution 



Hslwood Horticultural Training pnvate nonprofit organi- 
Center, Inc. zation 



Prince George's PIC, Inc. 



ABCO (A::tion for Boston 
Ccmainity Oevelopnnent) 



Vtefli s Technical Institute 



Boston Technical Center 



SOA 



camunity-based organization 



private educational institu- 
tion (nonprofit) 



private educational institu- 
tion, camiinity-based organi- 
zation - subsidiary of EOIC 



on-the-job training N/A 



adults SyDuth 



occiipational classrocm fixed mit price with prinHrily woren 18»- 
training less than 50% holdback 

for placement 

occupational classrocm fixsd unit price with prirarily women 18+ 
training less than 50% holdback 

for placemait 

occupational classroan fixed unit price with rrentally retarxied 
training, on-the-job less than 5(K holdback 
training for placarent 



on-the-job training N/A 



jpore adults than 
youth 



raredial basic educa- fixed unit price with adults & yxith 
tion, occupational less than 50% holdback 
classroom training, on- for placorent 
the- job training, work 
experience or try out 
erploynant, YEC curricu- 
lun, job-search assis- 
tance 

occupational classroan fixed unit price with woren 
training [non-tradi- less than 5(3% holdback 
tional (grc^ued)] for placement 



occupational classroom fixed unit price with 
training less than 50% holcback 

for placement 



general 



Bristol County 



SER/Jobs for Progress, Inc. conrunity-based organization 



fW Bristol Couity Bristol Ccnnuilty Col lege 



public educational Institu- 
tion 



occupational class- 
room training, ESL 
(adult); YEC curric- 
ulun, job placement 
(ioith) 

remedial basic educa- 
tion 

occupational class- 
rocm training 



fixed unit price with 
less than 50% holdback 
for placarent 



limited English 
adults; at-risk 
youth 



cost reinixirssment adults & youth 

fixed unit price with 
less than 50% holcback 
for placenent 
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state sw 



Narre of Pnovicfer 



W Bristol Ccxfrty J^eph Vlleno Associates 



^ype of Provicfer 



private for-profit 
orgfanization 



W Frarfclin^tepshire Fj^nklfrHfenpshire &plpyiBTt SM 
and Training Consortiim 



''A Brockton 



MA Brockton 



Brockton 



HD Jefferson- 
Franklin 



HD Jefferson- 
Franklin 



MD Kansas City 



Brockton Area PIC, Inc. SEW 



Il£/Th8 W}rk Correction 



Brockton Public Schools 



Jefferson College 



M) Kansas City Genesis 



M) Kansas City Urtan League 



Missouri Division of 
£iiplpyment Security 



mion 



Ptfalic educational 
institution 



State Ehployment Service other ptfalic 



agency 



Training Provided 



Target Population 



general 



^^or placsront 

ocaiMtional class- N/A 
rxxm training, on-the 
job training, basic 
«icatton, pre-erplpy- 
ment training, vork 
experience, tryoit 
QTplpjmert 

rarodial basic edbca- N/A 
tion, occMpational 
classnoon training 

training, ccst reiobui^eiHit 
SMPported work, 
with supervision by 
the program 

tion, work experience 
or try out erplpyment, 
TEC curriculuTi 



general 



(ex) offencfers 
(alternative sen- 
tencing) 



y^uth, dropouts and 
at risk 



public educational 
institution 



private educational institu- 
tion (nonprofit; 



comunity-based organization 



other public agency 



^^^^^^ for placeiEnt 

forplacsTEnt 

jo^searcbasslstana: fixed u,it price with iouth 
IJobCliJj) rare tfHn 5(K tolAack 

for placaiBrt 

S^S.«!"> f''^'"«t price with general 
joo-ssarch assistance, rare than 50% haldack 
custanized training for p Wrt 
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state SOA 



Kens of Provider 



T>pe of Provider 



Training Provided T>pe of Contract 



Target Population 



HO Bates-Vernon 



MO Bates-Vernon 



MO Bates-Venicn 



MO Butler-Wright 

HO Butler-Wright 

HO Butler-Wright 

W Albuquerque 

M4 Albuquerque 



State Dept. of Education 



Missouri Valley Hjtwi 
Resources Development 
Corporation Stteidiary of 



public educational 
institution 



union 



West Cfentral Missouri RDC conaaiity-tased organization 



S. Central Missouri Econo- 
mic Opportunity Corporation 



State Dept. of Elementary & 
Secondary Edbcation 



Easter Seal Society 



Career Services for the 
Handicapped 



Youth DevelopiEnt Inc. 



conmunity-based organization 



public edLjcational 
institution 



private nonprofit 
organization 



other ptilic agency 



private organization 
nonprofit 



rennedial basic educa- 
tion» occupational 
classroom training, 
(individual referral) 

remedial basic educa- 
tion» occupational 
classroom training, 
on-the-job training, 
icrk experience or try 
out erplqyment, YEC 
curriculun, job-search 
assistance 

remedial basic educa- 
tion, ocojpational 
classroom training, 
on-the-job training, 
work experience or try 
out eTplcf^ment, YEC 
curriculun, > job-search 
assistance * 

YEC curriculun, 
on-the-job training, 
job-search assistance 



occupational class- 
room training 
(remedial basic ed. 
under 8% funds) 

job-search assistance, 
on-the-job training 



on-the-j:b training, 
job-search assistajxe- 
conseling 

remedial basic educa- 
tion, work experience 
or try out eiploiment, 
YEC curriculun 



cost reintxirsaTHTt 
(reinixjrsenant for 
instructional costs) 



adults StyDuth 



fixed uiit price with 
more than 50% holdback 
for placement 



cost reinixnrsanent 
fixed unit price with 
irore than 50% holcback 
for placement 

fixed unit price with 
less than 50% holcback 
for placement 



fixed unit price with 
less than 50% holcback 
for placement 

cost reinixirsement 



adults & youth 



fixed unit price with 
more than 5(3% holdback 
for plccement 



adults & youth 



cost reinrixjrsement 



youth & adults 



youth & adults 



Jmiicapped adults 



adults with dis- 
abilities 



ciropouts ind at risk 
in school 
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state SCA 



Name of Provider 



Type of Provider 



Training Provided Type of Cdntract 



Target Population 



^M Albuquerque 

W Balance of State 
W Balance of State 



Work Uhlimited 

SER-Jobs for Progress 
State Dept. of Education 



other ptilic agency (closely 
linked with SCA) 



cornuiity-based organization 



^M Balance of State Div. of EhployiEnt Security 



public educational insti- 
tution 



other public agency lint'^ed 
with SQA 



NY Saratoga 
NY Saratoga 
NY Saratoga 



NY m( Partnership 
(Alleghany) 



NY SWiY Partnership 
(Alleghany) 

NY Syracuse 



Greater Saratoga Charter of 
Ccmrerce 

SaratogaAferren fiOCES 



Worldifl^ide Educational Services 



private nonprofit 
organization 

puilic edbcational 
institution 

private for-profit 
educational institution 



JanEstown Boys' and Girls' 
Clii) 



(Cattaragus County) Ocean 
School District 

Educational Opportunity Cfenter 



private nonprofit 
organization 



public educational institu- 
tion 

other public agency (EDC is 
nonprofit corp., but is 
part of SUNY, Div. of 
Special Prx3grams) 



cost reirrtxii'saTEnt 
(really a financial 
agrearent, not a 
contract) 



cost reiirbursement 



cost reirrixirsement 



adblts and youth 



adults and youth ' 
Hispanic focus 



disabled youth. 
rtDstly iTBitally 
retarded 



cost reinixjTsement general 



remedial basic educa- 
tion, occupational 
classnxm training, 
on-the-job training, 
job-search assistance 

on-the-job trainirg 
only and only in 
Santa Fe County 

remedial basic 
education, work 
experience or try out 
erplofynent 

rarodial basic educa- 
tion, occupational 
classnxm training, 
on-thfj-job training, 
work experience or try 
art erployrent, YEC 
curriculun, job-search 
assistance 



on-the-job training cost reifrfaursoiEnt adults & youth 

oxupaticnal classroom cost reiirbursaTBit adults & youth 
training 

occupational classroom fixed unit price with prirrarily women 18f 
training less than 50% holcback 
for placement (30%) 

YEC curriculun, job- fixed unit price with out-of-school yojth 

search assistance more than 50% holcback (drxDpouts) 
for placanent 

occ^wtional classroom cost reintxirsement adults & youth 
training 

remedial basic edjca- cost reirrbursement adults (general) 
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state 


SQA 


Name of Provider 


T>pe of Provider 


Training Provided 


T>pe of Contract 


1 

1 

t 

i 

Target Population ' 


NY 


Syracuse 


Syracuse City School District 


ptijlio educatioral 
institution 


occupatiaal classrooni 
training 


fixed unit price with 
more than 50% holdback 
for placement 


adults BtyDuth 


NY 


Syracuse 


Ehpire Vision 


other - private erplpyer 


on-the-job training 


Pays half salary for 
tins In training: 
placenent expected but 
not required. 


adults 


NY 


Yorkers 


Yorkers Rehab. Center. Inc. 


ccmnfiity-ba^ organization 


other - transitional 
supported oiplo^ment 


fixed unit price with 
less than 50% holcfcack 
for placement (20%) 


mentally 111 


NY 


Yonkers 


Yorkers PIC 


SDA 


on-the-job training 


fixed init price with 
less than 50% holdback 
for placarent (20%) 


adults &y3uth 


NY 


Yorkers 


Youth Services Agency 


other public agency 


YEC curriculim (also 
incorporates work 
experience or try out 
enplp>TOnt) 


fixed unit price with 
less than 50% holcback 
for placement (20%) 


teenage dropouts 


NC 


Central ina 


North Carolina Dept. of Labor 


other public agency 


on-the-job training 
adult/iojth 


cost reiriburserant 


general 


NC 


Centra lina 


Mitchell Camunity College 


ptfclic educational 
institution 


occupational classroon 
training, also indiv- 
idual referral, class- 
rocm training - basic 
skills assessment 


cost reiirtxirsement 


general 


NC 


Central ina 


Enploymsnt Security CCnmission 


other public agency 


intake eligibility, 
on-the-job training, 
individual referral 


cost reiirtxirsaiEnt 


general 


NC 


Gaston County 


Gaston College 


public educational 
institution 


classroon training, 
basic skills, ocojpa- 
tional classrocm 
training, accept indi- 
vidual referrals 


cost reiirbursement 


general 






' Cdunty Schools 


public educational 
institution 


pre-cnp./wori; nsturity 


cost reiirbursement 


in-school youth 


NC 


Gaston County 


Gaston-Lincoln Area 
Mental Health 


other pjblic agency 


transitional and 
st^jported €nplp>iTCnt 


cost reiirbursement 


mentally ill 
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state SOA 



Name of Provider 



Type of Provider 



Training Provided Type of Contract 



Target Population 



NC Winston-Salen 



OH Cleveland 



OH Cleveland 



OH Cleveland 



NC Winstcn-Salem Experiment in Self-Reliance 
NC Winstcn-Salem Urban League of Winston-Salan 
NC Wake-JohTSton SDA 



NC Wake-Johnston Association for Retarded 
Citizens 

NC Wake-Johnston Wake County Schools 



ccrmunity-based organization 



conrunity-based organization 



SQA 



private nonprofit 
organization 

public educational institu- 
tion 



Cleveland Hcmetcwi Program comninity-based organization 



ITT Career Center 



Fenix Circle Enterprises 



OH Crawfordl/Wyandot Tri-Rivers Vocational School 



OH Crawforcl/Wyandot ^rawford County urr!':e of the 
SOA 



private for-profit 
organization 

private for-profit 
organization 



public educational 
institution 



SOA 



OH Southeast Ohio Ejiplo>TCnt Service 
(Belmont-Washington) 



other public agency 



oca4.vational classroom H/A 
training, on-the-job 
training, job-search 
assistance, YEC curri- 
culun, work experience 
or try out erployiEnt 

raredial basic edjca- cost reitrfxjrsemait 
tion 

occupational classrocm cost reinbursarHit 
training 

on-the-job training, N/A 
job-search assistance 

on-the-job training cost reirrixirsement 



YEC curriculun 



cost reirrfxjrsement 



in-school yojth 
yoith & adults 



youth & adults 

wanen, youth & 
adults 

adults & youth 
mentally retarded 



in-school youth, jrs. & 
srs. not going to 
college 



job-search assistance fixed unit price with youth & adults 
more than 50% holcbadc 
for placement 

intake & assessment . ost reinbursement general 



rorredial basic educa- fixed unit price with youth 
tion, occupational less than 50% holdback 
classroom training for placement 



pre-flrployment, try cost reinbursement 
out erployrent, tutoring 
in basic skills 

basic education N/A 
remediation, job-search 
assistance, on-the-job 
training, referral to 
individual classrocm 
training, occupational 
classroom training 



in-school youth 



general, residents 
of Crawford County 



eligibility deter- 
mination 



cost reinrbursement general 
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state SDA 



Nam5 of Provider 



Type of Provider 



Training Provided Type of Gxitract Target Population 



OH Southeast Ohio Guemsy County 
(BeTtTOTt^feshington) 



OH Southeast Ohio Noble County Office 
(Belrrmt^feshington) CAP Agency 



OH Southeast Ohio Belmont County 
(Belront-Washington) 



OH Tnirbull County Trurixill Busing College 



OH TruTbull County ATES Technical Institute 



OH Tanbjll County Trurixjll County Joint Voca- 
tional School 



other public agency 



ccmunity-based organization 



cost reinfairsement 



cost reinburseTBTt 



on-the-job training, 
referral to individual 
classroan training, 
training 

on-the-job training, 
referral to individtal 
classroan training, 
occ^jatlonal classroan 
training 

camuiity-based organization on-the-job training, cost reinbursarent 

referral to individtal 
classroan training 



general, residents 
of comty 



general, residents 
of comty 



general, residents 
of cotrrty 



private (for-profit) 
edjcational institution 



private (nonprofit) 
edxational institution 



public educational 
institution 



occupational class- 
Txxm training 



occupational class- 
Txxxn training 



occupational class- 
room training, 
remedial basic educa- 
tion 



fixed unit price with 
less than 5(K holdback 
for placement 

fixed mit price with 
less than 50K holdback 
for placement 

fixed unit price with 
less than S(K holdback 
for placement 



adults &>wth 



adilts & yDuth 



adults &>wth 
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Appendix C 

AVERAGES OF INDEPENDENT VARIABLES USED IN QUANTITATIVE ANALYSIS 



State Policies Averages 

Use of model (number of standards for which model used) 6.61 
Whether adjustment procedures specified in policy 

(yes = 1, no = 0) ,44 

Emphasis on exceeding standards in 6% policy 

[Index of four State policies: Percentage of incentives 

received for marginally exceeding standards (weight = -.0098). 

Whether cap performance (weight = 0*72) • Number of standarcjs 

that must be exceeded (weight = 0.03).] -.25 

Use of 6% for hard to serve 

(Whether the State set aside a pool of 6% funds for serving 

hard to serve, whether the State requires a portion of 6% 

funds to be used for hard to serve, or whether 6% funds are 

exempted from performance standards if they are used for 

the hard to serve.) ,37 

Whether expenditure rates incorporated into 6% award 

(yes = 1, no = 0) ,33 

Adults Youth 

State policy for serving client group 

(Whether the State established a policy for serving the 

client group, whether service to the group is integrated 

into calculating incentive funds, or whether the client 

group is identified as a priority group.) .37 

Welfare recipients .45 .45 

Dropouts ,32 .35 

In school ,04 

Minorities ,34 .37 

Females ,23 .23 

Other barriers to employment .42 .42 

Over 55 years old .17 
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Adults Youth 

Incentive Weight on Standards {%) 
Adults 

Entered-ei^ployment rate 12.3% 

Wage at placement 10,6 

Cost per entered employment 18,1 

Welfare entered-employment rate 14,6 

Youth 

Entered-employment rate 18,6% 

Positive-termination rate 9^7 

Cost per positive termination 16,1 

PIC and LEO Characteristics Average 

(Not = 1, Somewhat = 2, Quite = 3, Extremely = 4) 

PIC influence on program design 2.81 

PIC influence in contracting 2.78 

LEO influence on progvam design 2.09 

LEO influence in contracting 2,13 

PIC and LEO concern about performance 

Relative to standard set by State 2.98 
Relative to other criteria 2.51 

PIC and LEO concern about expending funds 2.85 



Local Environment 

Unemployment rate (PY 86) 7 43 

Percentage of families with incomes below poverty level (1980) 9.73 

SPA Procedures (Director Sample) Adults Youths 



Enrollment criteria (proportion of activities for which 
criteria used): 



No specific criteria 


.18 


.18 


Education or basic skills criteria 


.40 


.43 


Judged more likely to complete program 


.36 


.31 


Previous work history 


.18 


.28 


Referred from other agency 


.18 


.25 
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Percentage of OJT participants selected first by employer 

Whether SDA established procedures to recruit and enroll 
client group (yes = 1, no = 0): 

Welfare recipients 

Dropouts 

In school 

Minorities 

Other barriers to eiri;)l oyment 
Ages 14 to 15 
Ages 55 and older 



SDA Procedures (MIS Samp ip) 



Percentage of participants receiving funding from other 
programs or other JTPA sources 

Expenditures in other JTPA programs as percentage of 
Title 1 1 -A 78% expenditures 

Percentage of expenditures in supportive services 
Proportion of program activities provided by 
SDA itself 

Public schools (including community college) 
Employment service and other government agencies 
Community- based organization 
For-profit organization 



Percentage of expenditures in performance-based contracts 
Terms of performance-based contracts (weighted by 
percentage of expenditures in performance-based contracts) 
Wage rate 

Proportion of full payment for placement 
outcomes 

Whether terms of performance-based 
contracts vary 



A verage 
16.3 

Adults Youth 

.75 .58 
.52 .77 
.69 

.39 .38 
.68 .69 
.30 



Average 

27.4 

58.5 
10.2 

.42 
.59 
.20 
.45 
.31 

Adults Youth 
24.2% 23.5% 

103.2 
11.8 
22.0 



Percentage of participants or irminees participating in 
program activities: ^ 

Job-search assistance {including 

pre-employment/Work maturity) 24.4% 
Basic skills 

Classroom training in occupational skills 31.8 



Adults Youth 



23.4% 
7.0 10.4 



16.8 

30.0 14.5 



Work experience 2 g 

Try out employment jg j 

Other 



12.1 
16.1 

3.9 6.4 
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Appendix 0 

SOA STRATEGIES RR AMEVIfG PERRRKANCE GCftLS 



SPA Goals 



SOA Strategies for Achieving Perfonnance 6m1s 



SOA Perfonrance in PY 86 
PerfoniBnce Award 




SCA#1 

Exceed by Meet par- 

10% to get ticipant 

naxinun $ needs 



SOA #2 
Exceed 
{prids, 
saiB $) 



Lock good 
to rest 
of world 



Balance long-tenn services 
for sens with direct job 
placaiEnt for large nuiters. 



Decide vtei to enroll 
participants in other programs 
tonBximize outputs. 



Design low-cost services. 
Increase OJTs (also a 
response to the econony). 
In-school YECs. Assure 
success by design of corponents. 



Build in high expectation in 
perfomence-based contracts. 
Negotiate unit costs based on 
line item bud^ to ensure 
cost-effect 1v0)ess. 
Vary wage expectations by 
occupational training arec to 
nBximize the level of each. 



Don't enroll OJT client until 
begins work. 

Don't enroll offencters until 
placed. 

Retain responsibility fo.' 
placaiEnt. though trainirg 
contractors help. 



.064 



1.96 



.109 



Expand individual referrals 
for classroan training. 
Don't do OJTs in tight labor 
nnrket. 



Monthly management meetings. 
Intensive review of client 
level problems and corrective 
action. 

Recent change fran maximizing 
nurbers to increasing intensity 
of services. 
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SCAGcals 



SCA Strategies for fehleving Perfonrence Goals 



SPA Perfomance in PY 86 



Perfonrance Other 



Service Design 



Client Targeting Goals 



Hanaqenent Practices 



Perfonnanoe /(ward 
as%of as % of 78% Stanckrxis 
Stancfard Allocation Hissed 



SQA#4 
Exceed ++ 
($, be#l 
in State, 
MhichnBdns 
largest 
aMard) 



Stretch $ Heavy enptusis on services 
to serve of short duration such as 
more people job club. 4-wsek OJTs. 

4-hair YECs; 10(% f ixecHnit- 
price, paid at placement. 



Freeze enrollments for all but 
targeted groups in incentive 
fonmla when necessary. 



Split-yaar strategy: first 
"run like crazy'' to take care 
of perfontance goals, then hope 
they have enough time and 
mon^ left ever to operate a 
service program fos* part of the 
year. 



(Averaged 
across 
standards) 
1.81 



.098 



SCA #5 
Meet ($) 



Find jobs, 
TCet needs 



Increased OJT (PIC likes). 



Planned versus actual in 
evaluatic^ reports (not 
including standards measures) . 
Discontinue nonperformers 
in subsequent year. 
Adopted YECs done by another 
SDA. 



1.74 



.143 



S0A#8 

Heet (stay Corply with 

out of State 

trouble) mandates 



Custanized training for State 
corrections department (high 
vage). 



Specialized contracts for 
handicapped youths; use of 
C80s to reach minorities. 



Rely on proven providers and 
experiment only with small 
amounts of ma:>ey. 



1.74 



.021 



SCA #7 
Exceed or 
meet 



Hininnl job-search assistance 
program, enrolling only those 
vto succeed (to inprcve wage 
outcomes, enroll only sihose 
Vfho get jobs above a fixed 
wageminimun). 



Refer high-risk clients to 
job-search assistance. 
Contracts with CBOs to serve 
dropouts and ^^icapped. 



Build requirarents into 
contracts and monitor heavily. 



1.6 



.034 



AMP 
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Appendix D [contlnuedl 



Sm Goals 



SPA Strategies for Achieving Perfomance Goals 



SPA Pgrfonrance in PY 86 



Perfonrance Other 



Service Design 



Client Targeting Goals 



Management Practices 



PerfoniBnoe Award 

as % of as % of 7©5 Standanis 

Stancferd Anocation Hissed 



S0AI8 

Meet More jobs. Develop YECs to attain YPTR. 

(pride, PR, less velfare. Use SVP guidelines to detennine 

' will be $) help aiployers djraticn of training. 



Guidelines for cost- 
reintxjrsanent contracts above 
stancfards level. 
Track perfonrance and push 
providers; discontinue those 
that cfan't perform. 



(Averaged 
across 
standards) 
1.66 



S0AI9 

Exceed ($, Stretch $ 

declining to serve 

allocation) more people 



SOA #10 

Exceed all Leverage 

standards other funding 

(PIC goal) sources 



Choose providers based on 
exceeding in the past and 
ability to deliver low-cost 
services (nultiple funding 
sources). 

Perfomance-based contract with 
m MdDoxk for plaosment at 
specified mfniuun wage. 
Centrali;^ed intake. 
They pusli providers, wonder 
if it's worth it. 



1.59 



Continuity of service providers 
with proven track record, with 
renewal of contracts contingent 
on continued high perfonrance. 
Perforronce-bosed contracts 
with 20% holcfcack for training- 
related placemsnt at specified 
wage. 

Careful plannirg to 'Txjild" 
contrast requirements that 
translate into success. 



1.57 



.070 



FRir 
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Appendix D [continued) 



SPA Goals 



SPA Strategies for Achieving Perfonnance Goals 



SCft Perfonrence in PY 86 



Perfomance Other 



Service Design 



Client Targeting Goals 



Managemsnt Practices 



Perfonnance Award 

as%of as % of 78% Standanis 

Standard Allocation Missed 



SCAIU 

Meet (stay PIC: produce 

out of trained 

trai)le) workforce 



Find veys to work with clients 
till they ARE ready. 
Sliding scale for OJTs. 
Sequence classroan and OJT. 



Focus on serving adults (the 
need is greater). 



If wage or cost look narginal, 
cream. 

Pay larger percentage of PUP 
for higher wages. 



(Averaged 
across 
standanis) 
1.45 



.033 



S0A#12 
Exceedf 
($t prida) 



Be one of Provide limited support 

tne best, services (save money to serve 

stretch $ more people) . 

Leverage other funds (e.g., 
education) for training. 



Pass along standards to 

counties, monitor. 

Don't allow OJTs under $4,25. 



1.44 



SUV #13 

Exceed ($, (success = 

daclining exceeding the 

allocation) stancfards) 



1005C fixed-unit-price 
contracting (payrent contingent 
on average wage rather 
than mininun) . Pass along 
incentive funds to providers 
based on their model- 
adjusted perfonnance. 
Use standards as part of 
rranaganent strategy*. 



1.39 



.043 



SCA#14 
Exceed ($ 
declining 
allocation) 



Meet varied YECs as a safety net. 
needs at Provide varied services to meet 
lew cost needs; eiployer outreach to 
irrprove wage. 



Overrecruit, allow urmotivated 
to fall out. 
Enphastze screening, 
(assessment contract with 
Goodvill). 



Expect the most that's 
reasonable from contractors, 
given history & resources. 



1.37 



.133 
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Appendix D [continued) 



Sm Goals 



SPA Strategies for Achieving Perfomance &>a1s 



SQA Perfomance in PY 86 



PerfonTBTce Other 



Service Design 



Client Targeting Goals 



Hanagemsnt Practices 



Perfomance Award 
as % of as % of 7a% Stancfarxk 
Stancfard Anocaticn Hissed 



SCA#15 

Heetall. Prove can do Stancfards have forced than to 
©oceed one veil, stay out run 2-tier program in which 
($) of trouble white rales go to OJT and get 

good veges. others go to class- 
room. 

6% ironey for adult rerediatfon. 



Overdo outreach so that 
applicants have to be 
persistent. 



Run a "tight ship", don't waste 
taxpayer money, differentiate 
fronCETA. 

$500 tonus to staff if meet or 
exceed standards. 



(Averaged 
across 
standards) 
1.34 



SQA#18 

Meet 

(pride) 



SCA#17 
Maximize 
award ($) 



Meet Fund cheaper courses. 

eiployer Shorter OJTs. 

needs Do rorediation before clients 

are enrolled (with non-Title II-A 

78% funding). 



PIC: keep 
irrproving, 
low cost 



Push for higher wage jobs. 
To inprove cost, use cheaper 
providers. 



1.33 



.031 



Nominally corpetitive RFP 
process, attrition of lowest 
perfonrers; perfonranoe-based 
contract for for-profit 
providers, cost-renrfxjrsed for 
schools; 



1.32 



.039 
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Appendix 0 [contirmf] 



SPA Goals 



SPA Strategies for Achieving Perfonnance Goals 



PerfoniBnce Other 



Service Design 



Client Targeting Goals 



ManagaiBit Practices 



SPA PerfoniBnce in PY 86 



PerfoniBnce A«rd 

as%of as % of 78% Standanls 

Stancfard Allocation Hissed 



SQA#18 
Exceed ($) 
or meet 
(pride) 



Serve the To attain >aith standards. 

disadvantaged including cost: reduce in- 
school prog^*anming and 
substitute a quick and 
inexpensive "Life Skills" 
training (motivated solely by 
standards and adopted with 
considerable reluctance by a 
staff that would rather be 
delivering significant 
services). 



(Averaged 
across 
standards) 
1.29 



.045 



SQ\#19 

Exceed ($, Earn respect; 

declining Serve need/, 

allocation) erploysrs 



Wien difficulty sending most 
in need, increase the erphasis 
on counseling. 



SQA has backup responsibility 

for placement. 

All cost-reifrfxirsement 

contracts. 

^bny services provided in-house 
in coiprehensive center. 
Contract out services to hard- 
to-serve groups. 



1.26 



.104 



SOA#20 
Meet 



Offer range Offer a mix of sen^ices; put 
of services pressure on OJT and job search 
to meet needs to operate cheaply so they can 

serve most in need with more 

intensive services. 



Assess clients carefully to 
ensure appropriate ratch of 
training. 



Follow individual participants 
to rreke sure they succeed or 
are referred to another 
ccnponent. 



1.25 



.041 
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Appendix D [continuecQ 



SDA Goals 




SDA Strateqies for Achievinq Perfonrance Goals 


SDA Perfonrsnce In PY 86 








Perfonrjince AwarxJ 


Performance Other 


Service Design 


Client Targeting Goals Hanagengnt Practices 


as%of as % of 78% Standarxis 
Standard Allocation Hissed 



SDA #21 

Meet (get $ Adjust to 

& stay out changing econ, 

of trouble) population 



Refine training to meet 
STployer needs. 
Establish YECs. 
Refuse to do "quick & dirty". 



New strategies for working with 
more disadvantaged. 



(Averaged 
across 
standards) 

Attention to the assessment and 1.23 
notching process. 
Insist on higher 
mininun wage for OJT. 
Vfcmen in (high wage) non- 
traditional occupations. 
Involve staff in corprehensive 
planning. 



.044 



CEE 



SDA #22 














Marginally 
exceed ($) 


Coordinate; 
train for 
good jobs 


Do v<hat th^ are doing an>way. 
Leverage funds through 
contracts with schools and 
other providers with 
multiple funding sources. 


Try to increeise minimim 
specified wage at entry to 
$5.00. 

Move toward perfonmiKfr 
based contracting. 


1.23 


.044 


AEBR 


SW#23 














Exceed the 

standards 

slightly 


To deliver 

appropriate 

training 


Use customized training (which 
can include OJT, as long as 
it's in a "new" job). 


Design contract requirements 
so that they include a 
significant margin for error. 


1.22 


.043 




SDAm 














Meet 
(can't 
spend $) 




Keep an eye on wages, 
occupations, and training 
deration. 


Hove slorfly and be conservative Use standards as "negotiating 
about whan you accept. tool" with contractors to push 

for lower costs 
Fixed-unit-price contracts 
(have raised costs as providers 
build in cushion). 


1.16 


.052 
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Appendix D [continuecQ 



SCA Goals 



SPA Strategies for Achieving Perfonmnce Goals 



SPA Perfomance in PY 86 



Perfonrence Other 



Service Design 



Client Targeting Goals 



fanaqement Practices 



Perfonrance Award 

as%of as % of 78% Standards 

Standard Allocation Hissed 



S0A#25 

Meet Serve mast 

in need 



Leverage workfare funds. 
Maximize liricages. 
Develop sound YECs. 



Enroll hardest in 8% programs. 



Cost-reitTfxjrsorent contracts, 
"Bookkeeping" respcnse to 
naximizing nsasured 
perfonrance, claim credit where 
it's due. 
Enroll third day. 



(Averaged 
across 
standards) 
1,15 



.047 



(AW, YPTR) 

but met by 

State's 

definition 

(within 

tolerance 

range) 



SDA#26 
Exceed by 
10% ($ and 
pride) 



Dropped enphasis on 14-15 year 
olds because "the model hurt 
them". 



Pass along requireients (higher 
U«n standards) to contractors, 
push hard and monitor 
continually (they are locked in 
to soTE large contractors that 
would be difficult to replace). 
Build in "standards + 10%" 
levels In their HIS to track 
perfonrance. 



1.12 



.053 



C/PT 



SDA #27 
Meet 

(money hard 
to spend) 



Placarents 



Tvo-ysar contracts, with 
perfonrance assessed after one 
year, shake out providers who 
aren't placerEnt oriented. 
Increasingly strict provisions 
of perfonrance-based 
contracts. 



1.11 



.000 



YEER 
(badly) 
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ApperxJix D [continuedQ 



SCA Goals 




SCA Strategies for Achievinq Perfonnance Goals 


SCA PerfomBnce In PY 86 1 








Perfonmnce A«rd 1 


PerfoniBnce Other 


Service Desiqn 


Client Tarqetinq Goals Kanaoercnt vractices 


as % of as % of 78% Stancfanis 1 
Standard Allocation Hissed 1 



S0AI28 

Heet ($ for Continue to 

neeting is operate good 

significant) program 



SDr.129 
Exceed or 
rreet 
(pride) 



SCAI30 
Meet (get 
out of 
tnxjble) 



Expecting trouble with >aith 

and will need to meet 7 of 7 to 

qualify, so th^ are 

"reluctantly" iroving toward 

adopting YECs. 

Also have deleted oqpensive 

training. 



Serve the 
local ccnnu- 
nity 



Try NOT to let standards drive 
the program; operate "business 
as usual" and use standards as 
a macfBnism for quality control 
after the fact. 



(Averaged 
across 
standards) 
1.09 



.063 



YEER. 
YPTR 



Pass along full risk to 
contractors via f ixed^jnit- 
price contracts with payrent 
1005J contingent on placement 
it a given minimm vege level, 
with higher payient for higher 
vege. Then work cooperatively 
with service providers to 
achieve goals (hired a 
PR/outneacK^job developnfait 
specialist to supplerent). 



1.06 



.018 



C/EE 



(Xality pro- OJT overperfonm to cushion 
gram meeting against riskier contracts, 
local needs If AC/EE slips, limit time in 

program to 6 months, then 4. 

Don't do job club or '^noney 

wasters". 



Accept all eligible applicants. 



Expand group of providers, keep 
an eye on those ih^t have 
failed. 

Bnphasis on $5.75 wage (rray 
have cost placonents). 



.093 



AEER 
C/EE 
YEER 
C/PT 
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DISTRIBUTION OF RESPONSES TO QUESTIONNAIRES 
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INTRODUCTION 



Three data collection instruments were developed for the quantitative 
evaluation of the effects of JTPA performance standards on the types of 
clients served, the types of services offered and the costs of those 
services. 

The first data collection instrument is the SDA director questionnaire. 
This questionnaire was mailed to the directors of all 610 SDAs in the United 
States. Telephone interviews were attempted for all SDA directors that 
failed to respond to the mail survey. A total of 530 SDAs provided responses 
to either the mail or telephone director questionnaires (a 87% response 
rate) . 

The second data collection instrument is the fiscal/MIS questionnaire, 
which was mailed to ar SDAs along with the director questionnaire. Because 
this questionnaire was not amenable to telephone interviewing, SDAs that did 
not return ti.e questionnaire were requested to do so by telephone, but no 
attempt was made to get the responses over the telephone. A total of 
454 SDAs (a 74% response rate) provided rosponi.es to the fiscal/MIS 
questionnaire. 

The third data collection instrument is the State perf rmance-standards 
coding form. Copies of state policies related to performance standards were 
obtained m all states and the District of Columbia. These policies were 
then coded by project staff using the coding form. 

This appendix presents 3 data collection instruments. The distribution 
of the responses to the 3 data collection instruments have been incorporated 
into copies of the instruments themselves. In addition, a table is appended 
to the fiscal/MIS questionnaire providing average values of some expenditure 
percentages, such as the percentage of total expenditures classified as 
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administrative expenditures. These a'^e averages of the percentages, not 
percentages based on the averages included in the questionnaire. 



The distributions presented are calculated for the SDAs that provided 
valid responses to each item. In cases where questions pertain to only a 
subset of SDAs or states, the distributions represent the percentage of SDAs 
with valid responses that responded to each category. To aid the reader in 
interpreting these responses, we indicate the size of the sample (denoted as 
n) providing valid responses for questions pertaining to a subset of SDAs or 
states. (In some cases, respondents did not respond to questions asked of 
them, so the sample sizes for related questions vary due to missing data.) 
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STUDY OF PY 86 JTPA I I -A 78% PROGRAMS 
SDA DIRECTOR QUESTIONNAIRE 



A. PIC Characteristics 



Al. In your opinion, how important is each of the following to the PIC? (n = 529) 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

HOW IMPORTANT TO THF PTr 





Not 


Somewhat 


Quite 


Extremel y 


a. Whether the SDA meets the needs 
of local employers 


1 ( 0.0%) 


2 ( 5.3%) 


3 (31.2% 


) 4 (63.5%) 


b. Whether the SDA meets the needs of 
those who are eligible for JTPA 


1 ( 0.2%) 


2 ( 5.9%) 


3 (32.7% 


) 4 (61.2%) 


c. Performance relative to SDA's 
standards, set by the state 


1 ( 1.3%) 


2 (18.1%) 


3 (46.0%; 


1 4 (34.6%) 


' a. Kerrormance relative to that 
of other SDAs in the state 


1 (12.7%) 


2 (41.5%) 


3 (33.2%] 


4 (12.7%) 


e. Performance relative to the 
national standards 


1 (13.1%) 


2 (41.7%) 


3 (3 1.9%] 


4 (13.3%) 


f. Performance this year relative 
to previous years' performance 


1 ( 2.1%) 


2 (27.3%) 


3 (48.0%) 


4 (22.6%) 


g. Expending the SDA's full 
allocation of funds 


1 ( 5.6%) 


2 (27.8%) 


3 (41.2%) 


4 (25.5%) 


A2. How if luential would you say the PIC is with regard to each of the following 
aspects of the JTPA program in your SDA? (n = 527) 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 




HOW INFLUENTIAL THE PIC IS ARnilT 


EACH ASPECT 




Not 


Somswhat 


Quite 


Extremel v 


a. The ty s of clients who are 
enrol'- in the programs 


1 (11.4%) 


2 (42.3%) 


3 (32.0%) 


4 (14.3%) 


b. The types of program services 
that are offered 


1 ( 0.9%) 


2 (18.3%) 


3 {39.5% 


4 (41.2%) 


c. The development of Youth 

Employment Competencies (YECs) 


1 ( 6.8%) 


2 (35.1%) 


3 (34. 5^;'- 


4 (23.5%) 


d. The type(s) of contracts used 


1 (13.3%) 


2 (36.6%) 


3 (27.8:<- 


4 (22.3%) 


e. The terms of the contracts 


1 (16.7%) 


2 (37.5%) 


3 (28.9- - 


4 (16.9%) 


f. The selection of service 
providers 


1 ( 4.7%) 


2 (12.1%) 


3 (31. e- 


4 (51.3%) 
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■ unaractensncs of t he Local Fleeted Official f s) 

* Jffi?ial(s)T°"{n^=^2T'"*^ ^^^^ °^ following to the local elected 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

HOW IMPORTANT TO THE 
LOCAL EIEHTED OFFICIAI (5), 



Not 



Somewhat 



a. Whether the SDA meets the needs 
of local employers 



1 ( 2. 



b. Whether the SDA meets the needs of 
those who are eligible for JTPA 1 ( 1 

c. Performance relative SDA's 
standards, set by the state 



d. Performance relative to that 
of other SDAs in the state 

e. Performance relative to the 
national standards 

f. Performance this year relative 
to previous years' performance 

g. Expending the SDA's full 
allocation of funds 



Quite Extreme] v 



2 (22.5%) 3 (41.2%) 4 (33.8%) 
2 (15.8%) 3 (39.8%) 4 (42.9%) 
2 (28.5%) 3 (42.8%) 4 (22.8%) 



1 ( 5 

1 (16.7%) 2 (43.1%) 3 (32.0%) 4 ( 8.2%) 
1 (18.4%) 2 (43.8%) 3 (29.1%) 4 ( 8 



1 ( 6.9%) 2 (33.7%) 3 (42.9%) 4 (16.6%) 
1 ( 7.1%) 2 (29.7%) 3 (38.3%) 4 (24.9%) 



■ ?nM^J^nrJho^''?'i'^''-^°" official (s) is/are with regard 

to each of the following aspects of the JTPA program in your SDA? (n = 525) 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

HOW INFLUENTIAL THE LOCAL ELECTED 









Not 


Somewhat 


Quite 


Extreme! v 




The types of clients who are 
enrolled in the programs 


1 


(22.8%) 


2 (42.8%) 


3 (25.8%) 


4 ( 8.6%) 


b- 


The types of program services 
that are offered 


1 


(15.3%) 


2 (43.8%) 


3 (27.7%) 


4 (13.2%) 


c* 


The dev^^lopment of Youth 
Employment Competencies (YECs) 


1 


(48.8%) 


2 (37.0%) 


3 ( 9.8%) 


4 ( 4.4%) 


d. 


The type{s) of contracts used 


1 

4. 


(33.3%) 


2 (37.4%) 


3 (19.5%) 


4 ( 9.8%) 


e- 


The terms of the contracts 


1 


(37.3%) 


2 (34.6%) 


3 (18.2%) 


4 ( 9.9%) 


f. 


The selection of service 
providers 


1 


(24.3%) 


2 (35.9%) 


3 (22.4%) 


4 (17.4%) 
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C. Performance Standards Issues 

CI. How influential would you say the performance standards are with regard to 
each of the following aspects of the JTPA program in your SDA? (n = 529) 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 



HOW INFLUENTIAL THE PERFORMANCE 
STANDARDS ARE ABOUT FAP.H /\<;PFrT- 







Not 


Somewhat 


Quite 


Extremely 


a. 


The types of clients who are 
enrolled in the programs 


1 (10.6%) 


2 (33.6%) 


3 (34. 


7%] 


4 (21.1%) 


b. 


The types of program services 
that are offered 


1 ( 5.3%) 


2 (24.6%) 


3 (47. 


6%] 


4 (22.5%) 


c. 


The development of Youth 
Employment Competencies (YECs) 


1 ( 9.3%) 


2 (24.8%) 


3 (39. 


5%] 


4 (26.5%) 


d. 


The type(s) of contracts used 


1 (15.2%) 


2 (25.8%) 


3 (33. 


7%] 


4 (25.4%) 


e. 


The terms of the contracts 


1 (14.0%) 


2 (21.6%) 


3 (36. 


8%) 


4 (27.5%) 


f. 


The selection of service 
providers 


1 (10.4%) 


2 (19.1%) 


3 (39. 


5%) 


4 (3i.0%) 


. Overall, how influential is each of the SDA's performance standards 
design of JTPA programs in your SDA? (n = 525) 

(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 


in the 






HOW 


INFLUENTIAL 


IS EACH STANDAi'^D: 






Not 


Somewhat 


Quite 


Extr(?mel v 


a. 


Adult entered-employment rate 


1 ( 1.7%) 


2 (14.5%) 


3 (42.. 


3%) 


4 (41.5%) 


b. 


Adult wage at placement 


1 ( 2.9%) 


2 (21.9%) 


3 (44.! 


5%) 


4 (30.7%) 


c. 


Adult cost per entered employment 


1 ( 3.0%) 


2 (23.2%) 


3 (42.! 


5%) 


4 (31.2%) 


d. 


Youth entered-employment rate 


1 ( 3.4%) 


2 (21.0%) 


3 (44.! 


5%) 


4 (31.1%) 


e. 


Youth positive-termination rate 


1 ( 1.3%) 


2 (15.6%) 


3 (45.: 


n) 


4 (37.3%) 


f. 


Youth cost per positive 
termination 


1 ( 2.9%) 


2 (25.1%) 


3 (42.: 


'%) 


4 (29.3%) 


g. 


Welfare entered-employment rate 


1 ( 2.9%) 


2 (21.3%) 


3 (43.] 


[%) 


4 (32.8%) 


h. 


Additional state standards 
(PLEASE SPECIFY:) 
















1 ( 5.7%) 


2 (14.2%) 


3 (42. J 


)%) 


4 (37.7%) 






1 ( 6.8%) 


2 (11.4%) 


3 (45. £ 


i%) 


4 (36.4%) 
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€♦1 Performance Stand ards Issupg (continued) 

stln&ll "''^ ^^termining any of your SDA's performance 

(Pl^ASE CIRCLE ONE NUMBER) 

Yes 1 (78.0%) 

PLEASE SKIP TO QUESTION C13, PAGE 7 <-- No 2 (22.0%) 

C4. How well does each model adjust for your SDA's circumstances? (n = 386) 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

ADJUSTMENTS MODEL MAKES FOR 

YOUR SDA'S CT RCUMSTANCF<;r Doesn't Annlv 
Model _£Qor Faii^^Jood_ Excellent Model No^^^I^H 

a. Adult entered- 

employment rate 1 ( 6.2%) 2 (28.0%) 3 (56.2%) 4 ( 8.3%) 0 ( 1.3%) 

b. Adult wage at 

placement l (18.0%) 2 (30.4%) 3 (43.3%) 4 ( 6.4%) 0 ( 1.8%) 

c. Adult cost per 

entered employment 1(10.1%) 2(24.7%) 3(53.1%) 4(10.1%) 0(2.1%) 

d. Youth entered- 

employment rate l (13.1%) 2 (31.0%) 3 (47.7%; 4 ( 5.6%) 0 ( 2.6%) 

e. Youth positive- 
termination rate 1 (15.4%) 2 (31.4%) ? (46.3%) 4 ( 4.6%) 0 ( 2.3%) 

f . Youth cost per 

positive termination 1 (14.4%) 2 (28.0%) 3 (47.8%) 4 ( 7.2%) 0 ( 2.6%) 

g. Welfare entered- 

employment rate 1 (10.5%) 2 (32.6%) 3 (48.3%) 4 ( 6.7%) 0 ( 1.8%) 

Yes 1 (48.3%) 

PLEASE SKIP TO QUESTION C7, PAGE 7 <- No 2 (51.7%) 

C6. What are they? 
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£i Performanc e Standards Issues (continued) 

C7. For PY 86, the percent welfare recipients served had a weight of -.252 in the 

adult entered-employment-rate model. Indicate whether you agree or disagree with 
the following statements about what this weight means. (n = 380) 
(PLEASE CIRCLE ONE NUMBER FOR EACH ITEM) 

, Tu • 1.4. • J- ^ . , Agree Disagree Don't Know 

a. The weight indicates that welfare 

recipients have lower entered- 

employment rates than do nonrecipients. 1 (78.9%) 2 (12.1%) 0 ( 8.9%) 

b. The weight indicates that the more 
welfare recipients served, the lower 

the standard will be. 1 (77.1%) 2 (12.0%) 0 (10.9%) 

c. The weight indicates that the more 
welfare recipients served, the easier 

the standard will be to meet. 1 (34.3%) 2 (53.3%) 0 (12.4%) 

C8. Did your SDA apply to the state for any governor's adjustments to its PY 86 
performance standards? (n = 401) 

(PLEASE CIRCLE ONE NUMBER) 

Yes 1 (12.0%) 

PLEASE SKIP TO QUESTION Cll, PAGE 6 <-- No 2 (88.0%) 

C9. For which standards did you apply for governor's adjustments? (n = 50) 
(PLEASE CIRCLE ALL THAT APPLY) 

Adult entered-employment rate 1 (34.0%) 

Adult wage at placementl 2 (58.0%) 

Adult cost per entered employment 3 (32.7%) 

Youth entered-employment rate 4 (38.8%) 

Youth positive-termination rate 5 (37.3%) 

Youth cost per positive termination 6 (36.0%) 

Welfare entered-employment rate 7 (32.7%) 



Additional state standards 
(PLEASE SPECIFY:) 



1 ( 4. 

2 ( 0. 
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Ci Performance Standards Issues (continued) 

CIO. On what basis did you ask for these governor's adjustments? (n = 47) 



Cll. Did your SDA receive any governor's adjustments to its PY 85 performance standards' 
(PLEASE CIRCLE ONE NUMBER) (n = 381) 



Yes 1 (22 

PLEASE SKIP TO QUESTION C13, PAGE 7 <-- No 2 (78.0%) 



C12. For which standards did you receive governor's adjustments? (n = 84) 
(PLEASE CIRCLE ALL THAT APPLY) 

Adult entered-employment rate 1 (48.8%) 

Adult wage at placement 2 (66.7%) 

Adult cost per entered employment 3 (48.8%) 

Youth entered-employment rate 4 (47.6%) 

/outh positive-termination rate 5 (48.8%) 

Youth cost per positive termination 6 (47.6%) 

Welfare entered-employment rate 7 (47.6%) 
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1 ( 6. 

2 ( 2. 

3 ( 1. 



^ — Pe«"forinance Standards t<<;iip<; (continued) 



if ? ^^^^ ^^^^^ PIC ""embers could benefit from 

(additional) technical assistance in performance standards is<;.iPO (n = 505) 

(PLEASE CIRCLE ONE NUMBER) ' 

Yes 1 (53.1%) 

PLEASE SKIP TO SECTION D <-- No 2 (46.9%) 

C14. What areas should this (additional) technical assistance cover? 



D. PY 86 Intake ProcPdiirAg 
Dl 



In PY 86, which of the following organizations conducted JTPA outreach and 

program-assignment activities for your SDA? (n = 530) °"^*^eacn and 

(PLEASE CIRCLE ALL THAT APPLY IN EACH COLUMN) 

Conducted Assigned Participants 

Outreach to Program Activities 

?. This SDA itself 1 (33,2%) j (7^ jo^j 

b. Public schools (including secondary 

schools, community colleges) 2 (69.8%) 2 (47.9%) 

c. Employment Service 3 (57.9%) 3 (27,2%) 

d. Other government agencies 4 (45.7%) 4 (16.6%) 

e. Community-based organizations, 

other non-profit organizations 5 (73.8%) 5 (43,8%) 

f. For-profit organizations 6 (42,3%) 6 (24.2%) 
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D. PY 86 Intake Procedures (continued) 



D2. How much difficulty, if any, did your SDA have recruiting a sufficient number of 

participants to expend its PY 86 funds? (n = 521) 

(PLEASE CIRCLE ONE NUMBER IN EACH COLUMN) 

Adults Youths 

No difficulty l (48.2%) 1 (29.8%) 

A little 2 (21.7%) 2 (23.6%) 

Some 3 (22.8%) 3 (23.4%) 

Quite a bit 4 ( 7.3%) 4 (23.2%) 



D3. Of those who applied and were eligible, about what percent actually received 
JTPA services from your SDA in PY 86? (n = 507) 

(PLEASE CIRCLE YOUR BEST ESTIMATE IN EACH COLUMN) 

Adults Youths 

Less t\'..i 25% 1 ( 3.9%) 1 ( 4.9%) 

?5% to 49% 2 (25.0%) 2 (16.8%) 

50% tc 74% 3 (37.7%) 3 (27.9%) 

75% to 90% 4 (23.5%) 4 (33.8%) 

90'^! or u re 5 ( 9.9%) 5 (15.6%) 
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D. PY 86 Intake PrncPrinrPQ (continued) 



04. Did your SDA establish specific procedfe to recruit and enroll anv of thP 
fol owing types of individuals as priority groups for PY 86 (e g esJablish 
contract requirements, special programs)? (n = 530) establish 

(PLEASE CIRCLE ALL THAT APPLY IN EACH COLUMN) 





Adults 


Youths 


Welfare recipients 


1 (74.5%) 


1 (57.7%) 


School dropouts 


2 (51.9%) 


2 (77.4%) 


In-school youth 




3 (68.7%) 


Minority groups 


4 (39.2%) 


4 (37.5%) 


Limited English-language 
speaking 


5 (21.5%) 


5 (18.9%) 


Handicapped 


6 (59.4%) 


6 (60.4%) 


Offenders 


7 (40.9%) 


7 (39.6%) 


Long-term unemployed 


8 (27.5%) 


8 (16.6%) 


Single heads of households 


9 (37.0%) 


9 (31.3%) 


Ages 14 - 15 




10 (30.2%) 


Other priority groups 


11 (30.6%) 


11 (17.7%) 


No specific procedures for 
enrolling any of above groups 


0 ( 7.0%) 


0 ( 6.2%) 



(PLEASE CIRCLE ONE NUMBER) 

Most were selected for JTPA participation 
in general and then were assigned to 

receive specific program services 1 (60.3%) 

Most were selected directly for 

specific program services 2 (39.7%) 
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06. In PY 86, what criteria were used to select eligible adult applicants for 
enrollment in each of the kinds of services provided? (n = 530) 
(PLEASE CIRCLE ALL THAT APPLY IN EACH COLUMN) 

SERVICES PROVTOED TO ADULTS: 



a. Took everyone who was eligible 
and willing to participate 

b. First come, first served in 
available slots 

c. Gav2 priority to those with 
a high school diploma or GEO 

d. Gave priority to those who scored 
above a given reading level 
(PLEASE SPECIFY GRADE LEVEL ) 7J. 

e. Gave priority to those who scored 
below a given reading level 
(PLEASE SPECIFY GRADE LEVEL:) hi 

f. Gave priority to those with a 
high score on other basic skills 
test (e.g., math level, overall 
grade level) 

g. Gave priority to those with a 
low score on other basic skills 
test (e.g., math level, overall 
grade level) 

h. Used occupational skills test(s) 

i. Used interest inventories or 
aptitude test(s) 

j. Gave priority to those with 
previous work history 

k. Gave priority to those judged more 
likely to complete the program 

1. Successful completion of other 
program activity 

m. Gave priority to those referred 
from other agencies or schools 



Basic 
Educ'l 
Skills 
Training 



1 
2 
3 



n. Employer selection 



7 
8 

9 

10 

11 

12 

13 
14 



36.0%) 
22.5%) 

1.: 

7. 

39.6%) 



2. 



41.3%) 
11.7%) 

28.9%) 
1.1%) 

16.0%) 

5.3%) 

16.2%) 
2.1%) 



Occupa- 
tional 
Skills 
Training 



1 
2 
3 



7 
8 

9 

10 

11 

12 

13 
14 



21.9%) 
30.8%) 
30.6%) 

34.7%) 

2.6%) 

30.4%) 

2.8%) 
50.2%) 

65.3%) 

5.5%) 

46.8%) 

20.8%) 
7.5%) 



OJT 



1 
2 
3 



7 
8 



V 



iO 

11 

12 

13 
14 



25.1%) 
27.9%) 
14.3%) 

8.7%) 

1. 

9.6%) 



1.9%) 
30.4%) 

47.0%) 

34.0%) 

45.5%) 

26.0%) 

16.2%) 
73.6%) 



Job 

Search 

Assistance 

1 
2 
3 



7 
8 

9 

10 

11 

12 

13 
14 
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D7. In PV 80, what cntena were used to select eligible youth applicants for 
%?r\lT')ioV}c^^.';^ °^ '^^"ds of services pro? ded? tsso) 
(PLEASE CIRCLE ALL THAT APPLY IN EACH COLUMN) ' 



a. Took everyone who was eligible 
and willing to participate 

b. First come, first served in 
available slots 

c. Gave priority to those with a 
high school diploma or GED 

d. Gave priority to those who scored 
above a given reading level 
(PLEASE SPECIFY GRADE LEVEL: ) 7J. 

e. Gave priority to those who scored 
below a given reading level 
(PLEASE SPECIFY GRADE LEVEL:) 6^ 

f. Gave priority to those with a 
ilic|h score on other basic skills 
test (e.g., math level, overall 
grade le/el) 

g. Gave priority to those with a 
low score on other basic skills 
test (e.g., math level, overall 
grade level) 

h. Used occupational skills tast(s) 

i. Used interest inventories or 
aptitude test(s) 

j. Used preempl oyment/work maturity 
competency test 

k. Gave priority to those with 
previous work history 

1. Gave priority to those judged more 
likely to complete the program 

ra. Successful completion of other 
program activity 

n. Gave priority to those referred 
from other agencies or schools 

0. Employer selection 



Basic 
Educ'l 
Skills 
Trainino 

1 (39.8%) 

2 (23.0%) 

3 ( 1.: 



4 ( 6. 

5 (40.9%) 



Occupa- 
tional 
Skills 
Training 


OJT 


Preempl oy- 
ment/Work 
Maturity 
Training 


1 (23.6%) 


1 (27.9%) 


1 (45.3%) 


2 (28.9%) 


2 (24.9%) 


2 (29.1%) 


3 (29.8%) 


3 (16.8%) 


3 ( 3.2%) 


4 (29.2%) 


4 ( 8.7%) 


4 ( 4.5%) 



5 ( 3. 



6 ( 2. 


5%) 


6 (29.4%) 


7 (44. 


5%) 


7 ( 4.0%) 


8 (11.! 


9%) 


8 (45.1%) 


9 (26.1 


3%) 


9 (58.1%) 


10 (13. ( 


5%) 


10 (16.8%) 


11 ( o.< 


)%) 


11 ( 8.5%) 


12 (14.3 


!%) 


12 (42.3%) 


13 ( 4.S 


1%) 


13 (23.8%) 


14 (24.2 


:%) 


14 (22.5%) 


15 ( 1.1 


%) 


15 ( 7.4%) 



7 ( 3. 

8 (27.5%) 

9 (41.9%) 

10 (18.7%) 

11 (29.4%) 

12 (39.1%) 

13 (28.5%) 

14 (18.5%) 

15 (62.3%) 



5 { 2.3%) 5 (10.9%) 



6 (ll.Ji%) 6 ( 2. 



7 (16. 

8 (14.9%) 

9 (33.2%) 

10 (50.4%) 

11 ( 4.3%) 

12 (17.0%) 

13 ( 7.9%) 



14 (26.2%) 

15 ( 4.9%) 
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D. PY 86 Intake Procedures (continued) 

D8. IF TESTS HERE USED AS A BASIS FOR SELECTING ADULT OR YOUTH PARTICIPANTS INTO 
PROGRAM SERVICES. Who chose to use these tests? 

(PLEASE CIRCLE ONE NUMBER) 

(n = 388) (n = 391) 
Adult Youth 
Service providers 1 (19.6%) 1 (18.2%) 
This SDA 2 (33.0%) 2 (32.7%) 



Both 3 (47.4%) 3 (49.1% 



E. Program Services 

El. Of the adults who received basic skills training in PY 86, about what 

percent also received occupationvl skills training or OJT? (n = 437) 
(PLEASE CIRCLE ONE NUMBER) 

Less than 25% 1 (37.8%) 

25% to 49% 2 (27.0%) 

50% to 74% 3 (18.1%) 

75% or more 4 (17,2%) 

E2. About what percent of PY 86 OJT participants were selected first by employers and 
then were referred to JTPA (or contractors) to determine eligibility? (n = 498) 
(PLEASE CIRCLE YOUR BEST ESTIMATE) 

Less than 10% 1 (52.8%) 

10% to 29% 2 (28.7%) 

30% w 49% 3 (11.0%) 

50% or more 4 ( 7.4%) 
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E. Prnqram Services (continued) 



£3. Were any Youth Employment Competency systems in place during PY 86? (n - 526^ 

(PLEASE CIRCLE ONE NUMBER) ' ^ ' 

Yes 1 (89.0%) 

FLEASE SKIP TO QUESTION Z12, PAGE 16 <- No 2 (11.0%) 



Hf ^ P*;^-e"'Ploynient/work maturity competency in place during PY 86? 

(it only a pre-employment or a work maturity comoetencv was in nl arp in dv na 

please answer about that component) („ I 473) ^ P''" 

(PLEASE CIRCLE ONE NUMBER) 



ERIC 



Yes 1 (92.6%) 

PLEASE SKIP TO QUESTION E7, PAGE 14 <- No 2 ( 7.4%) 



? oi Sr""J.°f Pre-employment/work maturity competency elements did 

a PY 86 participant need to be deficient in before being assessed as needina 

pre-employment/work maturity competency skills train itlS? (n = 423) 
(PLEASE CIRCLE INE NUMBER) 

Less than 15% 1 (17.3%) 

15% to 24% 2 (19.6%) 

25% to 49% 3 (18.0%) 

50% to 74% 4 (16.5%) 

75% or more 5 (9.7%) 

Varied among providers 0 (18.9%) 
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E. Program Services (continued) 



E6. What percent of the competency elements did the PY 86 participant hava U fiass in 
order to attain a pre -employment/work maturity competency? (n = 432) 
(PLEASE CIRCLE ONE NUMBER) 



Less than 25% l ( i 

25% to 49% 2 ( 1.4%) 

50% to 74% 3 (11.0%) 

75% to 84% 4 (40.7%) 

85% or more 5 (35.2%) 

Varied among providers 0 ( 9.3%) 



E7. Was a basic skills competency in place during PY 86? (n = 464) 
(PLEASE CIRCLE ONE NUNBER) 

Yes 1 (56.5%) 

PLEASE SKIP TO QUESTION ElO, PAGE 15 <-- No 2 (43.5%) 



E8. How far below reading-grade level did a PY 86 participant need to score 
before being assessed as deficient in basic skills? (n = 282) 
(PLEASE CIRCLE ONE mBER) 

Less than one grade 1 ( 6.0%) 

One up to two grades 2 (19. i%) 

Two grades or more 3 (21.3%) 

Below a specific grade 
level (PLEASE SPECIFY:} 

4 (20.9%) 

Varied among providers 0 (12.4%) 

Didn't use this criterion 9 (20.2%) 
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E. Program Services (concinued) 



Attained a specific grade 
^ el (PLEASE SPECIFY: ) 

8.0 



Varied among providers 
Didn't use this criterion 



E9. How many reading-grade levels did a PY 86 participant have to gain in order to 
attain a basic skills competency? (n = 281) yc«i" m oraer to 

(PLEASE CIRCLE ONE NUMBER) 

Less than 0.5 
0.5 to 0.9 
1.0 to 1.4 
1.5 to 1.9 
2.0 or more 



1 ( 2 

2 ( 5.3%) 
(28.5%) 
( 3.9%) 
(}3.9%) 



6 
0 
9 



( 8.S%) 
(15.7%) 
(21.4%) 



ElO. Of the youths who received basic skills training in PY 86, what percent 

also received occupational skills training, work experience, or OJT? (n = 383) 
(PLEASE CIRCLE ONE NUMBER) 

No basic skills offered 

Less than 25% 

25% to 49% 

50% to 74% 

75% or more 



0 ( 0.0%) 

1 (38.1%) 

2 (25.1%) 

3 (17.8%) 

4 (19.1%) 



Ell. Was a job-specific skills competency in place during PY 86? 
(PLEASE CIRCLE ONE NUMBER) 



Yes 
No 



(n = 461) 

1 (44 

2 (55.7% 
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E. Program Services (continued) 



E12. Some SDAs use funds from other JTPA programs to help fund services for 78% clients 
enrolling clients in both programs. What other JTPA programs, if any, provided ' 
joint funding for PY 86 78% programs in your SDA? 

(PLEASE CIRCLE ALL THAT APPLY) 

PLEASE 3KIP TO QUESTION E14, PAGE 17 <-- No others 0 (25.7%) (n = 530) 

JTPA 3% programs 1 (36.7%) (n = 381) 

JTPA 8% programs 2 (80.4%) 

JTPA Title II-B programs 3 (55.2%) 

JTPA Title III programs 4 (24.3%) 

E13. During PY 86, about what percent of 78% clients were dual enrolled in other 
JTPA programs? (n = 428) 

{PLEASE CIRCLE ONE NUMBER) 

Less than 10% 1 (57.5%) 

10% to 29% 2 (36.0%) 

30% to 49% 3 ( 5.1%) 

50% or more 4 (1. 



E14. What other programs or organizations, if any, provided resources to 
help supplement JTPA 78% adult or youth programs in PY 86--either 
direct funding or in-kind? 

(PLEASE CIRCLE ALL THAT APPLY) 

PLEASE SKIP TO QUESTION E16, PAGE 17 <-- No others 0 (22.3%) (n = 530) 



Welfare agency 


1 


(60.2%) 


Public schools 


2 


(72.9%) 


Other public agencies 


3 


(53.2%) 


Private sector 


4 


(32.8%) 


Other (PLEASE SPECIFY: ) 








5 


(18.6%) 
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E. Program Services (continued) 



E15. During PY 86 about what percent of 78% enrollees participated in programs 

that received resources from other programs or organizations' (n = 445 
(PLEASE CIRCLE ONE NUMBER) 

Less than 10% 1 (35.1%) 

10^» to 29% 2 (36.6%) 

30% to 49% 3 (15.1%) 

50% or more 4 (13.3%) 

E16. About what percent of the youths enrolled in your PY 86 78% programs had 

previously been enrolled in and terminated frJm JTPA 78% programs' (n = 502) 
(PLEASE CIRCLE ONE NUMBER) 

Less than 10% 1 (73.7%) 

10% to 29% 2 (21.7%) 

30% to 49% 3 ( 3.8%) 

50% or m^^re 4 ( 0.8%) 



long did they typically receive fireenrollment services such as ' counsel inq 
or assessment before they were formally enrolled into JTPA? (n = 516) 



E17. After individuals wore selected to receive PY 86 JTPA services, for h 

long did they typically re . > 
or assessment before they 
(PLEASE CIRCLE ONE NUMBER) 

For less than 3 days 1 (45 

For 3 to 5 days 2 (25.8%) 

For 6 to 10 days 3 (14.3%) 

For 11 to 15 days 4 ( 6.2%) 

For more than 15 days 5 ( 6.8%) 
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£♦ Program Services (continued) 



E18. The JTPA Annual Status Report (JASR) defines entered employment as entering 
full-or part-time unsubsidized employment at the time of termination. In 
reports of SDA performance on the entered-employment rate to the state, do you 
use this definition, or do you define entered employment in some other way? 

(PLEASE CIRCLE ONE NUMBER) (n = 526) 

Use JASR definition for state reports 1 (91.8%) 

Use another definition for state reports 2 ( 8.2%) 

E19. In repoi t of SDA per foi .nance on the entered-employment rate to the PIC, do you 
(use the JASR definition or do you define entered employment in some other way? 

(PLEASE CIRCLE ONE NUMBER) (n = 520) 

Use JASR definition for PIC reports 1 (88.7%) 

Use another definition for PIC reports 2 (11.3%) 



E20. IF USE ANOTHER DEFINITION FOR EITHER STATE OR PIC REPORTS, What is the 
definition that you use? 



F. Service Provider Arrangements 

Fl. Did you use any performance-based contracts for training services in PY 86? 

(PLEASE CIRCLE ONE NUMBER) (n = 525) 

Yes 1 (78.1%) 

PLEASE SKIP TO SECTION G, PAGE 25 <-- No 2 (21.9%) 
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F« Service Provider Arrangements ( cont i nued ) 



F2, Some SDAs set the same contract terms for all performance-based contracts while 
o\.her SDAs vary the contract terms among service providers. For adult PY 86 
programs, which of the following aspects (if any) of the performance-based contracts 
varied among service providers? (n = 411) 

(PLEASE CIRCLE ALL THAT APPLY) 

These aspects varied! 

Total amount of full unit price 1 (75,9%) 

Proportion of payment for final 

outcome 2 (58,9%) 

Intermediate outcomes for which 

p^irtial payments were made 3 (58,5%) 

Minimum wage rate 4 (59,1%) 

Whether payments were made 

for retention 5 (26,3%) 

Entered-employment rate goals 6 (52,3%) 

Requirements for service levels 

to specific client groups 7 (39,7%) 

PLEASE SKIP TO QUESTION F4, PAGE 20 <- None of the above. 

All terms were the same for all 
PY 86 performance-based contracts 
for adult programs 0 (11,9%) 



F3. On what basis did these terms of your performance-based contracts vary? 

(PLEASE CIRCLE ALL THAT APPLY) (n = 366) 

Characteristics of participant 

to be served 1 (60,4%) 

Type of service offered 2 (84,2%) 

Characteristics of service provider 3 (48.2%) 
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F. Service Provider A rranaem6nt.<; (continued) 

^TnJc '^^^^^'^^ ^^""-^ PERFORMANCE-BASED CONTRACTS VARIED, PLEASE ANSWER 

F4 THROUGH Fll WITH REGARD TO YOUR L ARGEST ADULT PERFOHANCE-BASED CONTRACT. 

F4. What payment points were included in the performance-based contracts for adult 
PY 86 programs? (n -• 411) 

(If these varied, answer for your largest adult performance-based contract ) 
(PLEASE CIRCLE ALL THAT APPLY) 



Payment points: 
Enrollment 

Intermediate outcomes before 
completion of training 

Completion of training 

Placement in ani^ job at termination 

Placement at termination in a 
traininq-relat.Pd job at a 
specified wage rate 

Retention in a job for a 
specified period of time 
(PLEASE SPECIFY PERIOD: ) 

1.3 months 



Other (PLEASE SPECIFY:) 



1 (69.8%) 



2 
3 
4 



(47.1%) 
(69.9%) 
(29.2%) 



5 (82.3%) 



6 (64.6%) 



7 (11.5%^: 



F5. 



For iMl PY 86 contracts, about what percent of the full payment was 
paid for placement, including retention outcomes? (n = 386) 
(If this varied, answer for your largest adult performance -based contract.) 
(PLEASE CIRCLE ONE NUMBER) 

Less than 25% 
25% to 49% 
505: to 74% 
75% to 99% 
100% 



1 (19.9%) 

2 (38.2%) 

3 (19.6%) 

4 (12.4%) 

5 ( 9.8%) 
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F« Service Provider Arrangement.'; (continued) 



F6. What mini mum wage rate was specified in the performance-basad contracts for 
adult PY 86 programs? (n = 370) 

(If this varied, answer for your largest adult performance-based contract.) 



$ 4.43 



F7. Was an entered-employmeni rate level specified as part of the performance-based 
contracts for aduU PY 86 programs? (n = 385) rrormance cased 

(If this varied, answer for your largest adult performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 



Yes 1 (72. 

PLEASE SKIP TO QUESTION FIO <- No 2 (27.8% 



F8. What was that entered -employment rate? 70,5% 

(If this varied, answer for your largest 
adult performance-based contract.) (n = 271) 



F9. Was a bonus paid to the contractor if the specified entered-employment rate 
was achieved or exceeded? (n = 307) 

(If this varied, answer for your largest adult performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 



Yes 1 (15. 

No 2 (84.4%) 

FIO. Did the performance based contracts for adult PY 86 programs specify required 
levels of service to different client groups? (n = 391) 
(If this varied, answer for your largest adult performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 

Yes 1 (38.9%) 

PLEASE SKIP TO QUEJTION F12, PAGE 22 <-- No 2 (61.1%) 
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F. Service Provider Arrangements (continued) 

Fll. Was a bonus paid to the contractor for meeting or exceeding the required 
levels of service to different client groups? (n = 248) 
(If this varied, answer for your iarqest adult performance-based contract.) 
(PLEASE CIRCLE ONE NUMBER) 

Yes 1 ( 8.1%) 

No 2 (91.9%) 



F12. For .youth competency-oriented PY 86 contracts, which of the following aspects 
(if any) of the performance-based contracts varied among service providers? 

(PLEASE CIRCLE ALL THAT APPLY) 

PLEASE SKIP TO SECTIOU G, PAGE 25<-- Doesn't apply: 
No competency-oriented performance- 
based contracts in PY 86 0 (33.8%) (n = 512) 

These aspects varied: 

Total amount of full unit price 1 (42.9%) (n = 340) 

Definition of final outcome 2 (20.4%) 

Proportion of payment for 

final outcome 3 (3K3%) 

Intermediate outcomes for which 

partial payments were made 4 (29.2%) 

Competency attainment rate 5 (23.3%) 

Required service levels to 

specific client groups 6 (20*4%) 

None of the above. All terms were 
the same for all PY 86 competency- 
oriented performance-based contracts 
for youth programs 7 (20.9%) 



F13. On what basis did these terms of your performance -based contracts vary? 

(PLEASE CIRCLE ALL THAT APPLY) (n = 290) 

Characteristics of participant 

to be served 1 (40.4%) 



Type of service offered 2 (54.1%) 

Characteristics of service provider 3 (30.7%) 
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F. Service Provider Arrangements (continued) 



^''nfSS^^ COMPETENCY-ORIENTED PERFORMANCE-BASED CONTRACTS VARIED 

(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ALL THAT APPLY) 

Payment points: 

Enrollment 1 (49,6%) 

Intermediate outcomes before 

completion of training 2 (31.3%) 

Completion of training 3 (44,6%) 

Attainment of competency 4 (59.4%) 

Other (PLEASE SPECIFY: ) 



5 (22.7%) 



F15. For competency-oriented youth PY 86 contracts, about what percent of the 
full payment was paid for the final outcome? (n = 223) 
(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 

Less t^^n 25% 1 (23.3%) 

25% to 49% 2 (32.7%) 

50% to 74% 3 (20.2%) 

75% to 99% 4 (13.9%) 

100% 5 ( 9.9%) 

F16. Was a com^etency attainment rate goal specified as part of the performance-based 
contracts for youth P/ 86 programs? (n = 228) MumidnLe oasea 

(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 



Yes 1 (71 

PLEASE SKIP TO QUESTION F19, PAGE 24 <- No 2 (28.9%) 
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F. Service Provider Arrangements 



(continued) 



F17. What was that competency attainment rate goal? (n = 157) 76 6y 

(If this varied, answer for your Taroest ' 

youth performance-based contract.) 



F18. To what type of competercy did this goal apply? (n = 128) 

(If this varied, answer for your largi^st youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 



Pre-employment/work maturity l (79.7%) 

Basic skills g (10.9%) 

Job-specific 3 ^ 



F19. Was a bonus paid to the contractor if that competency attainment rate goal 
was achieved or exceeded? (n = 215) 

(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 



Yes 1 (10 

No 2 (89 



F20. Did the competency-oriented performance-based contracts for youth PY 86 
programs require levels of service to different client groups? (n = 225) 
(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 

Yes 1 (34.7%) 

PLEASE SKIP TO SECTION G, PAGE 25 <- No 2 (65.3%) 

F21. Was a bonus paid to the contractor for meeting or exceeding the required 
levels of service to different client groups? (n = 133) 
(If this varied, answer for your largest youth performance-based contract ) 
(PLEASE CIRCLE ONE NUMBER) 

Yes 1 ( 6. 

No 2 (94. 
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6. SPA Characteristics 



Gl. What organization is the grant recipient for this SDA? (n 
(PLEASE CIRCLE ONE NUHBER) 

PIC 

Unit(s) of local government 
Non-profit organization 
Other (PLEASE SPECIFY: ) 



G2. What organization is th» administrative entity? (n = 521) 
(PLEASr CIRCLE ONE NUHBER) 

PIC 

Unit(.s) of local government 
Non-profit organization 
Other (PLEASE SPECIFY: ) 



G3. About what percent of the SDA s current staff worked for the 
(PLEASE CIRCLE YOUR BEST EJTIHATE} 

Less than 25% 
25% to 49% 
50% to 74% 
75% or more 



= 52i) 

1 (15.2%) 

2 (56.8%) 

3 (12.1%) 

4 (15.9%) 



1 (18.6%) 

2 (49.1%) 

3 (15.2%) 

4 (17.1%) 

CETA program? 
(n = 522) 

1 (30.1%) 

2 (26.6%) 

3 (21.5%) 

4 (21.8%) 
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a. SPA Characteristics (continued) 



G4. Who was the primary respondent to the questions of pages 4-26 of this 
questionnaire? (n = 486) 

(PLEASE CIRCLE ONE NUMBER) 

Di»'ector 1 (53,4%) 

Assistant director 2 (14.4%) 

Planning chief 3 (10.9%) 

MIS chief 4 ( 1,2%) 

Other (PLEASE SPECIFY: ) 

5 (10.1%) 



THANK YOU VERY MUCH FOR YOUR PARTICIPATION. 

Please use the enclosed postage -paid envelope to return this questionnaire to: 

SRI International, Employment and Tra 'i..-,' Research Program 
333 Ravenswood Avenue, Menlo Ps.v, A 94025 
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AVERAGE RESPONSES ON FISCAL/MIS QUESTIONNAIRE 
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STUDY OF PY 86 JTPA I I -A 78% PROGRAMS 
FISCAL/HIS QUESTIONNAIRE 



A. SPA PY 86 AVAILABLE RESO URCES FOR REGULAR TITLE II-A PROGRAMS 

Al. Total Title II-A allocation for PY 86 

for 78% programs: $ 2,325.415 

A2. Amount carried over into PY 86 from 78% funds in 

previous program years: $ 545 gsg 
(If none, please sntcr zero) ' 

A3. Total incentive funds received for performance 
in previous program years that are available. 

fQrJiseJn_Em: $ 18K063 

(If none, please enter zero) 



B. SPA PY 86 EXPENDITURES FOR REGULAR TITLE II-A PRORRAH<; 

Aote: Questions in this section refer to PY 86 expenditures from the 78% 
allocation, carryover funds, and incentive funds available for use 
in PY 86. 



Bl. Total PY 86 expenditures from the 78% allocation, 

carryover funds, and incent; ^ funds: $ 2.496.617 

B2. PY 86 administrative costs: $ 360.068 

83. PY 86 participant support costs (include needs- 
based payments, work-experience costs subject 



^ ^ ,^ wf^i^wi I V^UOUO OUUJC 

to expenditure limitations, and applicable 
employment generating activities): 



$ 276.342 



84. PY 86 actual expenditures for youth (including 

administrative expenditures): $ 1.050.417 
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B. SPA PY 86 EXPENDITURES FOR REGULAR TTTI F TT-fl D Dnr.DaMc (continued) 



B5. PY 86 required youth expenditure rate (i.e., 

percent of PY 86 expenditures thet were required 
to be spent on youth): 



36.5% % 



B6. PY 86 expenditures in performance-based contracts: 

ADULT 
$ 434.982 



YOUTH 
$ 297.077 



B7. 



For PY 86, what were the program services provided by the following types of 
organizations? ^ «.jk=^ 

(PLEASE CIRCLE ALL THAT APPLY IN EACH COLUMN) 



SERVICES PROVIDED: 



a. The SDA itself 

b. Public schools (including 
secondary schools and 
community colleges) 

c. Employment Service or 
other governmei.t agencie.*; 

d. Community -based organiza- 
tions or other non-profit 
organizations 

e. For-profit organizations 



Basic 
Eciuc'l 
Skills 
Training 



Occupa- 
tional 
Skills 
Training 



OJT 



Job Search 
Assistance or 
Preeniployment/ 
Work Maturity 
Traifii ria 



(24.3%) 2 (21.6%) 3 (57.7%) 4 (59.3%) 



(77.0%) 2 (84.8%) 3 (10.0%) 4 (56.1%) 

( 6.6%) 2 ( 8.6%) 3 (25.9%) 4 (38.2%) 

(37.0%) 2 (45.0%) 3 (39.3%) 4 (55.7%) 

( 9.3%) 2 (51.1%) 3 (35.0%) 4 (24.1%) 
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C. SPA PY 86 FXPFNnTTIIRF<: QN OTHER .1TPA PROGRAMS 



CI. PY 86 expenditures by SPA for each of the following programs (includinq 
adimmstrative costs): (If none, please enter zero) 



a. Title I I -A 8% programs: $ 147.186 

b. Title I I -A 3% programs: $ 62.22Q 

c. 1986 Title II-B programs: $ 1.044.508 

d. Title III programs: $ 163.302 



D. PERFORMANCE STANDARDS 



Dl. Incentive funds received for PY 86 performance : 
(If none, please enter zero) 



119.608 



D2. Final recalculated PY 86 performance standards , including a nv governors 
ad.iustments : ^ — 



a. Adult-entered-employment rate: 

b. Adult wage at placement: 

c. Adult cost per entered employment: 

d. Welfare-entered-employment rate: 

e. Youth-entered-employment, rate: 

f. Youth-positive-termination rate: 

g. Youth cost-per-positive-termination; 

h. Other (please specify any other 
performance standards thai were 
set for this SDA:) 



$. 
$. 



66.6% 



4.81 



4,107 



57.0% 



45.5% 



80.0% 



$ 3.382 
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E. PROGRAM SERVICES 



El. We would like to know how many individuals received various kind<' of 

rogram services, as listed in Question E2 on the following page/ BEFORE 
you answer that question, however, please give us the following 
information about how vou count who receives program services. 

a. Which of the following best describes how you count the numbers of 
individuals who received various PY 86 program services? 

(PLEASE CIRCLE ONE NUMBER) 

Individuals who received a program 

service at any time (i.e., individuals who 

received multiple services are counted in 
all categories) 1 ^73 

P>'i'"a>'y program service of individual 2 (15.0%) 

linX program service of individual 3 { 2.3%) 

Last program service of individual 4 ( 9.2%) 

Do the numbers of individuals who received various PY 86 program 
m^fJlTi^'^k 'SIT' °'" '° participants? 

Terminees . (17.8%) 

Participants 2 (82.2%) 



'J. 
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E. PROGRAM SFRVTCFS (continued) 



E2. 



Now that you've told us how you count how many individuals receiv-^d 
various PY 86 program services, please give us the numbers for each 
category listed below. {If none , please enter "0") 



NUMBER RECEIVING PY 86 .SFRVTr.F^; 



a. Job search assistance (including 
preemployment/work maturity 
training and direct placement) 

b. Classroom training in basic 
educational skills (including GED) 

c. Classroom training in occupational 
skills (including^vocational 
education, skill upgrading and 
retraining) 

d. On-the-job training 

e. Work experience 

f. Entry employment experience for youths 
(including try-out employment, 
school -to-work transition, and 
cooperative educ.) 

g. Other services 
(please specify:) 



Adults 



_24.7% 



7.0% 



31.7% 
_29.8% 



2.9% 



Youths 



24.8% 



10.7% 



15.6% 
14.3% 



12.3% 



16.5% 



4.( 



5.8% 



THANK YOU VERY MUCH FOR YOUR PARTICIPWION. 

Please use the enclosed postage-paid envelope to return this questionnaire to; 

SRI International, Employment and Training Research Program 
333 Ravenswood Avenue, Menlo Park, CA 94025 
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AVERAGE EXPENDITURE PERCENTAGES BASED ON THE FlfCAL/MIS QUESTIONNAIRE 

Average 

Expenditures as a percentage of available II-A 78% funds 81.1% 

As a percentage of total II-A 78% expenditures: 

Administration 14.6% 
Supportive services 10.4% 
Expenditures on youth 41.3% 
Expenditures on performance-bised contracts 24.9% 
Adult 
Youth 



25.1% 
24.0% 



Expenditures on other programs (Title I I -A 8% 

Title II-A 3%, Title II-B, Title III) 60.9% 

Incentive funds as a percentage of allocation 7.1% 



Note: These are average percentages and do not equal percen cages based on the 
averages presented in the questionnaire. 
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DISTRIBUTION OF ANSWERS ON STATE PERFORMANCE STANDARDS 

CODING FORH 
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STATE PERFORMANCE STANDARDS CODING FORM 



Adjustmen t Procedures 



Is the DOL model or the national standard used for each of the standards? 

(n = 51) 

National 

DOL Model Standard Other 


Adult entered-employment rate 


1 (84.3%) 


2 (15.7%) 


3 (0.0%) 


Adult wage at placement 


1 (84.3%) 


2 ( 7.8%) 


3 (7.8%) 


Adult cost per entered employment 


1 (84.3%) 


2 (13.7%) 


3 (2.0%) 


Youth entered -employment rate 


1 (84.3%) 


2 (13.7%) 


3 (2.0%) 


Youth positive-termination rate 


1 (84.3%) 


2 (13.7%) 


3 (2.0%) 


Youth cost per positive termination 


1 (82.4%) 


2 (15.7%) 


3 (2.0%) 


Welfare entered -employment rate 
(either model or ratio method) 


1 (84.3%) 


2 (15.7%) 


3 (0.0%) 


Is there a tated policy to allow for 
the model? (n = 51) 


additional 


adjustments oeyond 


Go to Bl - 


- MODELS NOT USED 


0 ( 5.9%) 




Yes 




1 (51.0%) 


Go to Bl - 


- No 




2 (43.1%) 



Are some adjustments granted across the board or must SDAs apply for 
all adjustments? (n = 26) 

Across the board 1 (46.2%) 

SDAs apply i (53.8%) 



So!o!Il^rLnf ^'""r^ procedures specified for applying for adjustments, 
bayond simply references to DOL TAG?? (n =26) 

Yes 1 (57.7%) 

Go to Bl -- No 2 (42.3%) 
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. IF YES, For which factors are adjustments considered? (n = 15) 
(Circle all that apply) ' 



Clients served 1 (86.7%) 

Services offered 2 (73.3%) 

Economic conditions 3 (93.3%) 



A7. Does thfi policy specify that adjustments should be requested at the 
planning stage? (n = 15) 

Yes 1 (60%) 

No 2 (40%) 
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B. Additional Standard^; 



What additional standards, if any, have been established? 
(Note: If state awarded a portion of incentive money based on whether 
the SDA met a state performance goal, it was coded ar. a standard even 
if the state did not call it a standard.) 



NONE 


0 


(64.7%) 


fn = 


51 ) 


Postprogram empl oyment/retention 


1 


(33.3%) 


(n = 


18) 


Post program earnings 


2 


( 5.6%) 






Service to dropouts 


3 


(16.7%) 






Service to welfare/WIN registrants 


4 


(16.7%) 






Service to minorities 


5 


(16.7%) 






Service to women 


6 


(11.1%) 






Service to handicapped 


7 


( 5.6%) 






Service to older workers 


8 


( G.6%) 






Placement of dropouts 


9 


(1] :;;) 






Placement of welfare/WIN 


10 


(11.1%) 






Placement of minorities 


11 


(11.1%) 






Placement of women 


12 


( 5.6%) 






Placement of handicapped 


13 


( 5.6%) 






Placement of older workers 


14 


( 5.6%) 






Economic development (e.g., placements 
in growth industries) 


15 


( 5.6%) 






Youth cost per entered employment 


16 


( 5.6%) 






Placement cf hard-to-serve 


17 


I 5.6%) 






Service to hard-to-serve 


18 


(16.7%) 






Weeks worked 


19 


( 5.6%) 






Youth wage at placement 


20 1 


[ 5.6%) 






Wage gain 


21 1 


[ 5.6%) 






Wei far job retention 


22 ( 


[ 5.6%) 






Expenditure rates 


23 ( 


[11. 1%) 






Entered employment for training activities 24 ( 5.6%^ 


) 
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C. Hard-to-serve policies 

CI. Are explicit policies established for serving specific hard-to-serve 
groups in JTPA, other than additional standards ? which groups? 
(Circle all that apply) 





Adults 


Youths 






C3 - NONE 


0 (41.2%) 


0 


(41.2%) 


(n 


= 51) 


Welfare reciDient<;/UTN 




1 


(56.7%) 


(n 


- 30) 


School JronoLit^ 


9 (Ad 09/ \ 


2 


(46.7%) 






In-school voLith. in aeneral 




3 


( 6.7%) 






In-SChool VOLlth nntpntipl Hv^nnmitc 




4 


( 3.3%) 






Mi nnri t V nrniinQ 

1 1 1 1 1 1 1 U Jr ^1 U U p O 


0 (oo,o/o) 


5 


(36.7%) 








c f on c\o/ \ 
0 \C\J.\j/o) 


6 


(20.0%) 






H^ndi caooed 


7 /fi'i '5<y^ 


7 


(63.3%) 






Offenders 


O y H D . / /o y 


8 


(50.0%) 






i.uiii| " term uneinp i oyeu 


die "70/ \ 

9 { 6.7%) 


9 


( 6.7%) 






Single heads of households 


10 (40.0%) 


10 


(50.0%) 






Ages 14 - 15 




11 


( 0.0%) 






Older adults 


12 (36.7%) 










Other priority groups 


13 (53.3%) 


13 


(53.3%) 






SDA option 


14 (23.3%) 


14 


(23.3%) 







C3. Other than incentive funds, has the state established a pool of 6% funds 
at the state level to be used for programs for the hard-to serve? (n = 51) 

Yes 1 (13.7%) 

Go to Dl -- No 2 (86.3%) 

C4. Approximately what percent of 6% funds are set aside for this state 
pool for serving tho hard to serve? (n = 7) 

48.9% 
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D. Incentive Poliry 

79.1% 

ff%l%m' °[„*i^4jj""tives are awarded proportional to the size 

65.2% 

Criteria for Qualifyi ng for InrentivP^ 

D3. Above what level must SDAs perform to qualify for incentives? (n = 48) 

Above standard 1 (68.1%) (n = 48) 

Above upper bound of tolerance range 2 (19.1%) 

Above lower bound of tolerance range 3(8, 
Above % of standards 4 ( 2.. 

Specify 2% 

uigh ranking among SDAs in state 5(2, 



D4. Which standards, if any, are in a core set of standards that are aiven 
priority in determining which SDAs qualify for incentives? ^ 
(Circle all that apply) 



NO CORE SET 


0 (81.3%) 


(n = 


Adult entered-employment rate 


1 (88.9%) 


(n = 


Adult wage at placement 


2 (22.2%) 




Adult cost per entered employment 


3 (22.2%) 




Youth entered-employment rate 


4 (33.3%) 




Youth positive-termi nation rate 


5 (33.3%) 




Youth cost per positive f.ermination 


6 ( 0.0%) 




Welfare entered-employment rate 


7 (77.8%) 




State standards, specify 







8 (11.1%) 



* Three single-SDA states did not specify these details in their policies, 
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D5. Minimum r imber of standards must meet tc oua'Jify for any incentives? (n = 47* 
Criteria for Determining the A, ^imt o f I.^/centives 

D6. In deterunir.g the weight for e.ach .standard, which of the followinq 
IS used for a majority of the funds: (p. = 47) 

Separate pools of funds for each standard 1 (46.8%) 

Percent above standard 2 (42.6%) 

Percentage point?; or dollars above standard 3 ( 6.4%) 

Other 



4(4, 

07. What is the weighl given to each standards in determining the amount of 
incentives? 



EQUAL WEIGHTS FOR EACH STANDARD 


1 


(57.4% 


) (n = 


Adult entered-employment rate 


16 


.9% 


(n = 


Adult wage at placement 


15 


.0% 




Adult cost per entered employment 


11 


.7% 




Youth entered-employment rate 


14 


.1% 




Youth positive-termination rate 


8 


.8% 




Youth cost per positive termination 


. 8 


.0% 




Welfare entered-employment rate 


15 


.4% 




Total of state standards 


10 


.2%* 





*29.2% for those with state standards 



08. What percent of the total incentive payment is received for marginally 
exceeding standards (i.e., for being eligible for incentives)? 

34.7% (n = 47) 



* One additional single-SDA state did not specify these details in its polic. 
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D9. Is there a threshold above which the SDA no longer receives additional 
incentives for improving performance on individual standards? (n = 47) 

1 (44.7%) 

Go to Dll No 2 (55.3%) 



DIO. What percent above the standard (or minimum qualifying level) is that 
threshold set? (n = 22) h j a icvci; ib tnat 

23.4% 



Dll. Is there a maximum number of standards that are considered so that an 
standards?" (^^^47) incentives for exceeding additional 

Yes 1(2 
Go to L)13 -- No 2 (97.9%) 



D12. What is that maximum number of standards? (n = 1) 



D13. Do SDAs receive extra bonuses for extraordinary performance' 

(n = 47) 

Yes 1 (14.9%) 



No 



2 (85.1%) 



iL^^^^^ competition among SDAs so that the amount received by one 
SUA depends on the parformance of other SDAs? (n = 47) 

Yes 1 (66.7%) 

No 2 (33.3%) 
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Additional Recuirements 

D16. Other tha n additional standards, are requirements about serving thp 
hard-to-serve incorporated into the incentive policy? Which groups? 

Adults Youths 



Go to 018 



-- NONE 


0 


(8i/.4%) 


0 (89.4%) (n 


Welfare recipients/WIN 


1 


(80.0%) 


i (80.0%) (n 


School dropouts 


2 


(60.0%) 


2 (60.0%) 


In-school youth, in general 






3 ( 0.0%) 


In-school youth, potential dropouts 






4 ( 0.07c) 


Minority groups 


5 


(20.0%) 


5 (40.0%) 


Limited English-language speaking 


6 


( 0.0%) 


6 ( 0.0%) 


Handicapped 


7 


(60.0%) 


7 (60.0%) 


Offenders 


8 


(60.0%) 


8 (60.0%) 


Long-term unemployed 


9 


(20.0%) 


9 (20 0%) 


Single heads of households 


10 


(40.0%) 


10 (fiO.0%) 


Ages H - 15 






11 ( 0.0%) 


Older adults 


12 


(40.0%) 




Other priority groups 


13 


(20.0%) 


13 (20.0%) 


SDA option 


14 


;.^j.o%) 


14 (20.0%) 



D18. Are expenditure rate requirements incorporated into the 
incentive policy? (n = 47) 

Yes 1 (31.9%) 

No 2 (68.1%) 
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D20. Is there a requirement that incentive funds must be used to serve 
the hard-to-serve? (n = 47) 

Yes, an 1 { 2.1%) 

Yes, :j5r-tially i ( 4.3%) 

No 3 (93.6%) 

D21. Are p; funds excluded fron: the evoluation of an SDA's performance 
if they are used to serve ihe hard to serve? (n = 47) 

Yes 1 (17.0%) 

No 2 (83.0%) 
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E- Sanction Policy 

El. Which standards, if any, has the state established as a core set of standards 
that are given priority in determi^iing whether SDAs fa^l their standards? 
(Circle all that apply) 

Go to E2 -- NO CORE SET 0 (83.3%) (n = 42*) 

Adult entered-employment rate 1 (85.7%) (n = 7) 

Adult wage at placement 2 {28.6%) 

Adult cost per entered employmont 3 (42.9%) 

Youth entered-employment rate 4 (4?. 9%) 

YoM.th positive-termination rate 5 (42.9%) 

Youth cost per positive termination 6 (14.3%) 

Welfare entered-employment rat^ 7 (57.1%) 

E2. Minimum number of standards that must fail to be classified as 
"failed to meet" in one year? (n = 41) 

3.1% 

E3. How does the state define failing two consecutive years? (n = 38) 

Miss all same standards 1 (21.1%) 

Miss some of the same standards 2 ( 7.9?^) 

Miss the same number of standards within 

the core set 3 (13.2%) 

Miss same number of standards, 

regardless of which ones 4 (57.9%) 

E4. Below what level is considered failure? (n = 40) 

Below standard 1 (52.5%) 

Below lower level of tolerance range 2 (40.0%) 

Below the upper level of tolerance range 3(2. 

Below % of standard 4(5. 
Specify 85% 



* Nine states did not have a sanction policy. Several aditional st^es did not 
specify specific items so that the number of cases with valid data varies. 
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OUTLINE OF TOPICS FOR ON-SITE DISCUSSIONS 
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OUTLINE OF TOPICS FOR ON-SITE DISCUSSIONS: STATE STAFF 
QUA- ITATIVE ANALYSIS 
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OUTLINE OF TOPICS FOR Off SITE DISCUSSIONS: STATE <TAFF 

QUALIT/tTIVE ANALYSIS 

ROLES AND RESPONSIBILITIES OF DIFFERENT STATE-LEVa ACTORS 

1. What are the relative responsibn ities of State legislature, SJTCC 
manbers, SJTCC staff. Governor, state JTPA administrative staff in 
initiating state policy, overseeing program impl ementati on> 
suggesting revisions to state policy, reviewing requests for 
adjustments, implementing incentive and sanction policies? 

2. vrhat is the frequency of SJTCC meetings, topics discussed, policy 



3. What are the particular interests of the Governor's office in JfPA 
policies? 

4. What are the responsibilities of SJTCC staff versus state JTPA 
administrative staff? 



SUWWRY VIEWS Ot^ PERFORMANCE STANDARDS 



1. Overall, how useful are the performance standards as measures of 
JTPA national program goals? 

2. Overall, how have the performance standards affected perceptions of 
the JTPA program by SDAs, employers, and the general public? 

3. How does the state view the relative importance of the different 
federal standards? 

4. What are some of the specific ways in which the existence of 
performance standards has furthered federal or state objectives for 



(Probe: e.g. emphasis on outcomes; emphasis on improving 
performance over time; emphasis on serving welfare recipients 
because of existence of AWEf standard; emphasis on cost 
effectiveness) 



5. What are soma of the specific ways in which the existence of 
performance standards has inter<'ered with or created incentives 
that make the realization of federal or state objectives more 
difficult to achieve? 

(Probe: e.g. incentive to cream; encouragement of superficial low 
cost interventions; encouragement of "over-performance"; too hard 
for PICs to understand mechanics of model or r chanics of incentive 
policy; YECs are meaningless; standards make ".t harJ to target 
hard-to-;erve) 
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1. 



6. What are your suggestions for Improving the way the perfomance 
standards system works? How would this change the Incentfvel 
operating on SDAs and/or service providers? incentives 

STATE VIEWS ON PERFORMANCt' STAffflARDS DESIGN 

How does the mere existence of the sev... federal performance 
standards Influence SDA behavior. Irrespective of the numerical 
levels that are set? What kinds of Influences result from the mere 
existence of the seven federal standards? 

(Probe: e.g. has existence of AWEE standard encouraged/discouraged 
service to welfare recipients? has the existence of AEE standard or 
AWE standard encouraged/discouraged services to Individuals or 
groups with particular employment barriers? has the existence of a 
cost standard causid SDAs to design only "cheap" services?) 

Do you have any suggestions for changing the number of standards 
standTrds^?"^ standards, deleting standards, combining 



2. 



3. How has the numerical level at which the federal standards are set 
influenced the behavior of SDAs? 

l^/?.^'' r.'^'^^ federal standards/national departure points too 
high? If yes, why is this a problem? Are they too low? If ves, 
why IS this a problem?) ^ 

4. How have SDAs been affected by the changes over t'me in the levels 
at which the national standards have been set? Wha»- are the 
positive and/or negative impacts of these changes? How have state 
^JiV^ l u'' P/'actices (e.g. for adjustments to SDA standards) been 
affected by the change's in nati-nal numeric performance standards? 

Do you have suggestions for changing the numerical levels (national 
standards or national departure points)? 

How did the state decide whether or not to Implement additional 
state performance standards? What standards were considered? What 
is the intent of any state perfonnance standards that have been 
implemented? How important does the state consider them to be, 
t-on+lw the federal standards? Would you like to see them 
adopted as federal ii:andards? 

How do the state- initiated standards influence SDA behavior? Do 
they influence who is selected to receive JTPA services? Do they 
influence what typos of services are provided? Do they influence 
the cost of JTPA services? Are these effects intentional or 
unintentional effects of the state standards? 

What do ycu_ think was the federal intention behind the development 
of the l.cai adjustment models? How effective have the models been 
in achieving these aims in practice? Is it a goca idea to have 



S. 

6. 



8. 
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tolerance ranges defined around the model -derived numerical 
standards? 



How did the state make the decision to use or not to use the DOL 
adjustment model? 

(Prubej what alternatives were considered?) 

Do you think the local adjustment models are able to respond to 
changes in program emphasis over timer? What are the problems/ if 
any# that arise* because the models are oased on data generated 
two years earl ier? 

Please describe shortcomings of particular local adjustment model si 
based on your experience. Have you developed any further 
adjustments to overcome these problems? 

Have federal changes to refine the local adjustment models from 
year to year been helpful or disrupt H/e? 

How did the state develop its policy regarding SDA requests for 
adjustments to the model-derived standards? What is that policy? 

(Probe: whether and on what basis adjustirtents can be requested; 
who makes the decision about granting adjustments; whether the 
state considered rranting adjustments for servina especially 
difficult clients) 

What has been the frequency of case by case adjustments? 

(Probe: on wSiat basis have adjustments been made? in response to 
what special circumstances? with what effec+ on numerical 
performance standards?) 

What is the relevant history for the case study SDAs? 

Other than developing additional state perfurmance standards, what 
policibs has the state developed regarding services to hard-to- 
serve individuals and groups? 

(Probe: has the state set asiue discretionary funds to promote 
projects for hard-to-serve clients? does the state have a stated 
policy for SDAs to target hard-to-serve groups* or give priority to 
hard-to-serve individuals? has the state encouraged SDAs to 
provide services to address basic skills deficiencies among 
adults?) 

What policies or procedures has the state implemented to ensure 
equitable service to significant population segments (e.g. age 
groupings* sex groupings* racial groups) as well as groups 
explicitly mentioned in the JTPA legislation (WIN registrants and 
high school dropouts)? How are these policies enforced? What are 
the consequences for an SDA that fai'^s to meet these policies? 
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17. What messages for SDAs did the state try to build Into it^ 
incentive allocation policy? « i^uua mto its 

it'to fl'n 1'^''^°'''"/"?^ standards are most Important? how bad is 
}° '"^^^ ^ particular standard? how good is ic to 
"overachieve" on a particular standard? if you can't meet all 
standards, how s. .uld you make the best of limited resources? etc 

What are the minimal requirsments that any SDA must mee. to qualify 
for an award? How would an SDA maximize its rewards? ^^^'i^V 

InZ^J^ 1^^*^ P''^^^^^ constrained by federal 

guidelines on the use of the 6% incentive funds? ^^^^^^f^^' 

It^nH Jlf ri^'i^'.r'* •'^"^^ °" exceeding rather than meetino 
f^ nrnnnV ^^^^ "'^ *° "'^^^ Standards for two consecutive years 
Should bt .h' ''^79^"^^^*^°"' that projects using Incentive funds 
Should be themselves included within the cost limiiations and 
performance standards monitoring system) 1 1 1 -axi ons and 

wou?dio:K2°chrt's?" ''''''' '^''"^ 
pe^utL*VevS:\^hie::s\yTts^ 

(Probe: How were Incentives allocated in response to PY86 

Wha'^°w'^rJh ^^'^^ ^^^^ *° ''^^^•'^^ ^'centive award? 

Whac was the smallest award? the largest award?) 

+uTl^'T^ ^^^"^ unintended effects of the incentive design? Is 
?u?ur1? ^°^''^^'^^9 changes to the Incentive design for the 

JJf the Intent behind the state's sanctioning policy? Whet is 
(Probe: what is the lavel at which the SDA must de-elop a plan to 

uS^lperfo^^a^e'^Lfg^Jt^^SoT^^^^ ^" ^^^--^"^ 

21. f'^^t the state's philosophy of technical assistance to SDAs 
in relation to SDA performance? ■s»-ance xo bURs 
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22. What types of technical assistance have been provided by the state 
or the federal government to SDAs In the following areas: 

0 reporting requirements 
0 performance standard-^ and the DOL model 
o state Incentives and sanctions policy 
0 how to target the hard-to-serve 

0 how to design Innovative and/or effective service 
programs 

0 how to design service provider contracts 
0 how to collect follow-up data 
0 other topics 

SDAs?*^''^^ °^ technical assistance have been most well received by 

23. What do you think is the federal intention behind the introduction 
of the planned post-program performance standards? 

24. What is your impression of the likely effect of introducing post- 
program standards? How is required post-program reporting 
currently influencing SDA behavior? What are the potential 
problems with the planned post-program standards? What are the 
benefits of the planned post-program standards? 

25. What role has the state played in the development or refinement of 
YEC systems within the state? 

(Probe: what is the level of YEC development within the state? 
what does the State see as its appropriate role in the process of 
implementing YEC reporting systems and YEC service delivery 
systans?) ^ 

What can be done to improve the meaningful ness of the YEC part of 
the performance stand-^ds? 

STATE VIEWS ON OTHER FEDERAL POLICIES AFFECTING THE JTPA SYS1B4 

1. How has the federal limit o i administrative expenditures influenced 
state policies and practics? How has it Influenced SDA behavior? 
How has it affected who g^^ts served by the program and what 
services are offered? Have you learned how to live with this 
federal regulation? How would you change it if you could redesign 
federal policy? ^ 

2. How has the federal limit on supportive services costs Influenced 
state policies and prixctices? How has it influenced SDA behavior? 
How has it affected who gets served by the program and what 
services are offered? 

(Probe: Have you learned to live with this federal regulation? 
How would you change it if you could redesign federal policy? Do 
you have any experience requesting waivers to supportive service 
cost 1 imits?) 
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3. How has the federal requirement for 40% expenditures on youth 
services Influenced state policies and practices? How has it 
influenced SDA behavior? How has It affected who gets servZ by 
the program and what services are offered? ^ 

r" ^^^'"^^ *° ""^^^ th^s federal regulation? 

How would you change it if you could redesign federal policy? Do 

pvno H.? ^"^ experience requesting adjustments to the 40% youth 
expenditure requirement?) yuutn 

4. How are you affected by federal reporting requi ranents for client- 
level outcomes and model factors?. . .for expenditure data?., for 
post-program outcome data? Have JTPA data collection requirements 
influenced state or local progra,.i design decisions? Are data 
colleccion requirements unnecessarily burdensome? [see also a. El 
below. J ^ 

5. What are the problems, if any, with the data required by the 

federal government? What are your concerns about data accuracy, 

H^;f nli^^r definitions, adequacy of model factors, follcwLp 

''*^°IL/''^''*^''^^^ W*^^* ^''^^"9^2 would you make in data 
collection practices if you could? Isee also q. E2 below.] 

OTHER STATE POLICIES AFFECTING THE JTPA SYSIEM 

1. What additional reporting requirements has the state imposed beyond 
those required by the federal government? What was the reason for 
these additional data elements? What has been their effect on SDA 
behavior? Have they had any noticeable impact on the types of 
services offered by SDAs or on the types of clients served? 

^* ^I^^JL^h ^^^.^ur\ ''^"'^ ^" designing and/or maintaining a 

state-wide JTPA MIS system? What effect has this had on: 

0 the frequency of feedback on SDA performance to the SDAs; 
0 ^.le frequency of feedback on SDA performance to the state, 
0 the state's ability to monitor who is receiving JTPA 
services; 

0 the SDA«s ability to monitor who is receiving JTPA services- 
0 the freq-iency of SDA feedback on provider performance; 
othe ability of the state and/or SDA to request ad hoc 
reports analyzing perf omri-ince to date. 

dllMng the'yeafr*' ''''''''''''' ''^'"^ monitoring SDA performance 

(Probe: what is the frequency of on-site monitoring visits? what 
types of reviews are made at an on-site visit? what Interim 

ot1f°?l';?:n'''*''''".f''' '"^^^^^''^^ ^•^^ ^-t-^ what lypel Of 
enS ?hl year?)° requested, if any, before the 
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, ! x^f*^^ policies regarding expenditure rates by its 
exoe'nditt'/"l^' reallocated if they are'not spent? Have 1 
expenditure rates been a problan for the state? How have state 
policies influenced SDA behavior? 

Has the state developed a policy regarding the proper form and 
content for a perf onmancs-based contract between an SDA and its 
service providers? Have there been any abuses in the performance- 
based contracts that have been developed by SDAs in the state? How 
have state policies in this area influenced SDA behavior? 

(Probe: what are desirable features of a performance-based 
contract, according to the state? what are undesirable ?eatures of 
a performance-based contract?) aturei, or 

How is state policy and practice affected by federal auditing 
Snn^r^f 4.^""^ .''f''^"* developments in federal policies about 
XXh contracting practices? How are SDAs 

atfected by these policies? ^ — 

ilTri 'If ^^"^^'^^ policies in this area influence who is 
served by the program and what services are offered? What changes 
would you make in federal policy if you could and why would you 
make these changes?) ' 

What are state goals and objectives regarding coordinating JTPA 
/o% resources with resources from other JTPA pools, and other 
programs? What coordination efforts have actually been 
!wn:f®'I*®'^^.."°*' ^^""^ '^^^^^ coordination efforts influenced the 
f!'? enrolling in JTPA programs, or the types of 

services that have fesen offered with JTPA funds? 

Ini^i'ltld^^pf."* ^^^^ P''°9''^'" influenced by state- 

initiated welfare refonr, measures? How have state designed welfare 

JnA^ru influenced the types of services^ffered by 

tvSpI nf .Hon? ^t^^'^designed welfare initiatives influenced the 
types of cl ients sen/ed by SDAs? 

How are state JTPA policies and practices influenced by federal 
policies and congressional concerns regarding welfare reduction, 
including <lr-ft legislation for a variety of work/welfare refonn^? 

(Probe: -. this influencing who is being served by the JTPA 
program ar what services are being provided?) 

Do you anticipate closer/ ties in the future between JTPA programs 
and welfare reform objectives? H«"arams 
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STATE REACTIONS TO SDA PERFORMANCE IN PY 1986 



1. 



2. 



furina PvUf^T ^1 the range of SDA performance in your state 
(JuMng PY1986? What changes would you like to see in SDA 
performance in the future (in terms of individual SDAs or state 
averages)? 

What do you think about the range of services that were provided by 
SDAs in your state in PY1986? What changes would you like to see 
in service patterns in the future? 

3. What Jo you think about the intensity of sep/ices provided by the 
SDAs in your state durinq PY1986? What changes wo'-^d you like to 
see in the intensity of srvices in the future? 

4. What do you think about the patterns of individuals who received 
services across the SDAs in yoiT state during PY1986? What chances 
would you like to see in who jiets served in the future? 

5. For each of the above desired changas, will performance standards 
help or hinder the state in trying co encourage SDAs to iriake thes- 
changes? 
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OJTLINE OF TOPICS FOR ON-SITE DISCUSSIONS: SDA RESPONDENTS 

QUALITATIVE ANALYSIS 
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Askt Name of Respondent 
Title 

Previous Experience with CETA 
Lengti of Job Experience with SDA 
^ytept 9f TnvolvCTent with Perform a nce Stands rri? 



Putllpe pf T9Plx:s for On-Slte m^cu^^in nst 5>DA Rec;p9n ^ont?T 

QuaTltative An;^lY«:tc: 

SDA 

Administrators Chief PI anner, SDA Service Evaluation 

(Director* Program Develop- Intake Provider Staff, Fiscal 

~ Pei>titY PlrfirtOP) fnftnt ^t^ff Director Monitors Mrq gt^ff officer 

A. SDA Goals, OMecttveSr Priori ties 

1. Tha Roles of Different Local Actors x x 

What are the roles and concerns of: 

• PIC members 

• Local elected officials 

• SDA staff 

Specifically what is their Interest in policies 
on priorily clients, sen^ice design, contractor 
performance requi ranents, strategies to maximize 
SDA rewards* 

What is their Involvement in operational izing 
client selection, contractor selection, contractor 
monitoring, assessing SOA performance* 



2. The Effects of Local Factors on SDA Performance 
OMecttves and Practtcos 

Size and Characteristics of JTPA Eligible Pool 
and Applicant Pool 

Types of Local Job Opportunities 

• Types of Jobs 

• Stability of Employment 

• Wage Scales 

Geographic Size of SDA 

Level of SDA<s $ Allocation 

Organized Political Influence or On Behalf of 
Particular Constituencies (e.g., various ethnic 
^f^ougs, welfare recipients, non-Engl ish speaking, 
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SDA Topic Outline 



121 



SDA 

Administrators Chief Planner, SDA Service Evaluation 

(Director, Program De>/elop- Intake Provider Staff* Fiscal 

— — ' Deputy Ptr^rtPP P^gnt Staft Director Monitors MTS Staff Officer 

A. 2« (continued) 

For each of the topics, the questions are: 

• Hof has this factor influenced the design and 
evolution of SDA performjince objectives 

• Hof has this factor Influenced SDA or service 
provider actions that affect clients served* 
services provided* or service costs 



en 



3. . Smssary ii&ts on Perf onDance Standards 

Overall, hon useful are the performance standards 
as measures of desired perf orriance. 

Do the planned post-program standards represeat an 
Improvement over termination 'based standards. 

Overall* hw have the performance standards 
affected perceptions of the JTPA program by PICs* 
local elected officials* service providers, and the 
general public* 
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4. SDA Goals on Clients Served XXX X 

Does the SDA have priorities for serving 
Individuals with particular characteristics or 
members of particular target groups* 

How did you decide what the priorities were (e*g** 
PIC concerns, state requi ronents* local nsed 
assessment). 

Have priority groups changed from year to year. 

What are the hardest types of clients to serve 
under JTPA. What are the barriers to serving them. 

What are your strategies* If any, for serving T A 

clients perceived as "hard to serve." i V 

What was the role of performance standards In the 
development of SDA goals on clients served. 

Specifically* hew did the welfare entered employ- 
ment standard Influence SDA goals on service to 
welfare recipients. 
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SOA 

Administrators Chief Planner^ SDA Service Evaluation 

(Director^ Program Develop- Intake Provider Staffs Fiscal 

Deputy Dlrec^ior) ment Staff Director Monitors MTS Staff Officar 



A* 4« (continued) 



Ho4 did other federal or state policies Influence 
the development of SDA policies on clients served, 

Hw were SDA client priorities communicated to 
service providers. 



5. SDAGcals on Service Design 

What Is the SDA*s plan for allocating resources 
across different types of services* and ho* was It 
determined. 

Are certain types of services emphasized. Are 
others excluded. 

What was the role of performance standards In the 
development of SDA goals on service design. 

m How did other federal or state policies Influence 

^ the development of SDA goals on service design. 

Hw did the development of Joint funding linkages 
with other funding sources Influence the develop- 
ment of SDA goals on service design. 



6. SDA Developeaent of YEC System 

Does the SDA have YEC system In place. 

What types of YEC categories were used. 

Hw Is deficiency defined. Hew Is goal 
achievement defined. 

Ha# was the choice of YEC made. 

What was the role of the PIC and other groups. 

What was the role of performance standards In 
developing YECs. 
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SOA Topic Outline 



SOA 

Administrators Chief Planner, SDA Service Evaluation 

(Director, Program Develop- Intake Provider Staff, Fiscal 

— Deputy Director) ment Staff Director Monitor? HIS Staff Officer 

A* (continued) 

7. SOA Suggestions for Inpravlng the YEC System XX XX 

Changing the number of YECs 

Changing the Hay Deficits or Accomplishment Is 
Measured on Each YEC 

Changing the Number of YECs that Must Be Achieved 
Other 



B. Doveloptaont of SOA Perfonaance Oyectlves X 
* 1. SDA Perfonnanco Oyectlves 

What are SDA priorities and objectives reoardlngt 

t relative Importance of the different standards. 

0 the Importance of maximizing financial rewards. 

• desired performance levels on each standards 
and hew arrived at (e.g.. In comparison to 
last year, In comparison to this year, in com- 
parison to model-acl) usted standards. In com- 
parison to the national standard. In compari- 
son to other SDAs). 

Hew did local snplpyer needs Influence SDA per- 
formance objectives* 

Do you have local performance goals that are not 
adequately reflected by the federal or state 
performance standards. 

Does the SDA place different priority on the 
standards than does the state (as reflected In 
the Incentive awards). 

What are the trade-offs among performance stan- 
dards (I.e. areas where Improving performance on 
one measure tends to Impede performance on 
another measure). 




What Is your strategy for resolving trade-offs among 
performance areas. 



SOA Topic Outline 
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B* 1* (continued) 

much do each of tho follofing Influence SOA 
behavior: 

• maximizing SDA funding allocations through 
r^^axlmlzlng rewards 

• avoiding sanctions 

• furthering the SDA»s own priorities about 
performance 

• achieving or maintaining good community 
perception of the program 

HoK do these different Incentives differ In how 
they affect SDA behavior. 

Is there a conscious relationship between the 
SDA«s performance objectives and: 

• Its policies or goals on priority client groups. 

• Its policies or plans for design of Its service 
programs. 

• Its practices for assigning clients to 
^ particular services. 

O • Its practices for selecting and monitoring sor- 

^ vice providers. 

t the terms of Its service provider contracts. 

What does the SDA do to monitor Its own per- 
formance during each program year. 

What mid-course corrections are made# If any# If 
performance levels appear to be below the do si rod 
1 evel s. 

What do you think about your actual PY 86 per- 
fonmanco levels. What changes would you like to 
sou In the future. 

What do you think about the range of services you 
provided In PY 1986. What changes wcuU you like 
to see In the pattern of services provided In the 
future. 

What do you think about who you served In PY 1986. 
What changes would you like to see In who Is served 
In the future. 

For the abovor hcM will the performance standards 
constrain you, If at all# In making tho desired 
changes. 



SDA — 

Administrators Chief Planner, SDA Service Evaluation 

(Director, Program Develop- Intake Provider Staff, Fiscal 

Deputy Plregtpr) mm SUff Director Monitors MIS Staff Officer 
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SDA Topic Outl ine 
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SDA 

Admtnlstrators Chtef Planner, SDA Gorvtce Evaluation 
(Director* Program Develop- Intake Provider Staff* Fiscal 
. Deputy Director) mont Staff Director Monitors MIS Staff Officer 

B. (continued) 

* 2, SDA Strategies for laproving Performance f rcn XX XX 

Yoar to Year 

• If the SDA has had difficulty (or were to have 
difficulty) meeting one or more performance 
objectives, what was (would bo) their strategy 
for improving perfonmance, 

« How did (would) this affect the SDA designs for 
client selection, services offered, types of 
providers* provider contractual arrangements. 

• What Impact will the implementation of follc?f-up 
standards have on SDA performance strategies in 
the future. 



C. SDA Policies and Practices for Program Impleraentation X X 

1. Describe the flew of clients, and SDA versus 
contractor responsibilities for: assessment* 
eligibility determination, enrollment, assign- 
ment to services* placement* tenmination. 

2. Practices Affecting Client Selection X X 

What are SDA policies and practices regarding client 
out*^each : 

• Who does outreach. 

• Dc» some or all providers do their own 
oLt reach. 

• Wh&t liiethods are used, 

• Is special outreach used to generate 
applications from specific types of 
applicants (e.g.* youth, adults* more 
qualified* less qual if led) . 

What are SDA policies and practices regarding client 
Intake: 

• Who is responsible for Intake. 

• What are Intake criteria and procedures. 

• Who Is excluded from participating and 
for what reasons (basic educational 
deficiencies* motivation* particular 
aptitudes* too Job ready). 

What arc practices regarding enrollment: 

• What pre-enrollment activities are avail- 
able. 
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SOA 

Administrators 
(Oirector* 
Deputy Director) 



Chief Planner! 
Program Oevelop- 
ment Staff 



SOA 

Intake 
01 rfictor 



Service 
Provider 

Monitors 



Eval uatlon 
Staff, 

MIS Staff 



Fiscal 

Officer 



C, 2* (continued) 



I 



• Oo these activities vary by type of 
client* type of service received (e»g»> 
classroon training versus OJT)» 

• What happens to early dropouts from 
various components. 

Is there a conscious relationship between 
enrollment practices and SOA client service 
priorities* 

What are SOA policies and practices regarding 
assignment to services (i*e*» matching clients 
to services): 

• Hof is a client assigned to a particular 
service. 

• What is the effect of space avail abil ity» 
client interest* client performance on pre- 
enrollment tests. 

• Are clients ever assigned to multiple 
services. 

Is there a conscious relationship between 
assignment to service practices and SOA service 
design goals. 

Is there a conscious relationship between 
asslgnmont to service practices and SOA 
performance objectives. 

Hew much variation has there been from year to 
year in SOA policies and practices regarding 
client outreach^ Intake^ and enrollment in 
service. 

What impact has the existence of performance 
standards had on the design and evolution of 
these policies and practices. 



3, Practices Affecting Services Doliverod 

In advertising for/selecting service providersr 
what specifications does the SOA make 
regarding: 

e what types of services shall be offered 
(including supportive services). 



o 110 

ERIC 111 



SOA Topic Outline 



SDA 

Administrators Chief Planner, SOA Service Evaluation 
(01rector# Program Oevelop- Intake Provider Staff* Fiscal 
Oeoutv 01 r ector) ment Staff Oirector Monitors MT<: Staff Officer 

C* 3* (continued) 

• the specific content of services, 

• the expected duration of services* 

o the expected sequencing of services, 

• the volume of clients to be enrolled In 
each typo of service. 

Oescrlbe the range of youth and adult services 

to be provided according to SOA funding XX XX 

announcements (Including % In-school youth; 
% out-of-school youth progranimlng). 

Hotf much variation has there been over time 
In the overall service design since the 
beginning of JTPA. How much difference 
from C£TA. 

Are cirect placements permitted. Encouraged. 
Olscouraged. 

ni * What Impact has the existence of performance 

standards had on the design and evolution of the 
00 SOA»s practices for offering services. 



4. Practices Affecting Typos of Service Providers and 
Reflation of Provider Performance 

Horf are providers recruited and selected: X X 

• How many contracts are there. 

• How are contracts differentiated (e.g.# by 
service, by client* type, by geographic 
area). 

• What Is the mix of organizational types of 
providers (e.g.* public educational Insti- 
tutions* other publ 1c agencies* community- 
based agencies) 

Are contractors selected through a competitive 
bid process. 

• How competitive Is the selection process. 

• Oo the sarne providers tend to apply each year. 
^ P • Are there service providers who have stopped 

''^ bidding or whose contracts have not been renewed. 

HoH many of the current providers were also 
service providers under CJETA. 

O ~~ 
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SOA Topic Outline 
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SOA 

Admlntstrators Chief Planner, SOA 
(OlrectoPi Program Oevelop- Intake 
Oeoutv Director) ment ^fp^ff Director 



Service Eval uatloa 
Provider Staff* Fiscal 

MPnUors mis staff officer 



C» 4* (continued) 



How are contracts negotiated (who participates 
In decisions about contract terms, whax con- 
tract conditions are of most concern to SOAs) 

What do contracts require regarding: 
• different perfonnance payment points 
(e.g., enrol Iment, compl eti ons, entered 
employment, training-related placements* 
retained employment for "X" period) 
minimum required performance levels 
consequences of failure to meet minimum 
1 evels 

compensation rate for each payment point 
bonuses for high performance 
required client characteristics 
consequences of failure to meet required 
characteristics 



How do thesa contractual requirements vary 

• by provider 

• by type of service offered 

• by type of participant to be served (do 
any provisions vary for "hard to serve" 
participants) 

How do these contractual arrangements relate to 
SOA client service priorities. 

How do these contract requirements relate to the 
SOA«s cwn numerical performance standards. 

What are SOA monitoring practices regarding: 

• frequency 

• relation to contractual requirements 

• attention to Interim underperf onmance 
problems 

How has actual provider performance measured up 
on contractual requirements In the past. 

What have been the consequences for undor- 
perfonmers. For high performers. 



X X 
X 
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SOA 

Admtntstrators Chtef Planner, SDA 
(Dtroctor, Program Develop- Intake 
Deputy 0< rector) ment St^ff 01 rector 



C. 4, (continued) 



Service 
Provt der 

Monitors 



Eval uati on 
Staff, 

MIS SUff 



Fiscal 
Officer 



I 

o 



HG 



What is your opinion aboutj 

• the advantages and/or disadvantages of 
perf onnance-based contracting. 

• how SDA contractual r€'!u» 'ements and 
monitoring procedures ave influenced 
service provider actioi-s that affect 
clients selected, services provided, 
or service costs, 

• whether there has been a learning curve 
that will make future effects different 
from the past. 

What impact has the existence of performance 
standards had on the evolution of the SDA»s 
practices for selecting sen^ice providers and 
designing service contracts. 



5. SDA Practices and Procedures Regarding Placctoent 
and Termination 

Hew is placement defined for payment to providers. 

Hew is placement defined for reporting on JASR, 

Who is responsible for placing individuals on 
completion of training. 

When does termination from JTPA occur. 

What are the follow-up responsibilities of 
service provider versus SDA, 

What impact has the existence of performance 
standards had on SDA practices and procedures 
regarding placement and termination. 

Is there any conscious relationship between 
placetpent and termination procedures and SDA 
performance objectives. 
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SOA 

Administrators Chlof Plannorr SOA Service Evaluation 
(Olrector* Program Oevelop- Intake Provider Staff* Fiscal 
— Deputy Director) ment <;taff Director Monitors MIS <;taff Officer 

0« Understanding of Federal and State Porformanco Stan- 
dards Policies 

i. The Choice of Additional State Porformanco 

Standards XX XX 

Hcht do you understand: 

• the Intention of the state standards 

• the Intended relative emphasis on federal 
standards versus state standards 

• the Influence of additional state standards 
on SOA actions that affect clients served, 
services provided* or service costs 

• whetlier there are any unintended effects of 
additional state standards 

Do you hdve any suggestions for changing the 
number of/content of federal or state standards. 



2, State Alterations to/Excoptlons to the Local 
A((justiaent t4odels 

m 

Hof (!o you understand: 
*^ • the Intention of state a((justments to the model 

• SOA experience with these alterations or 
3(0 ustments 

• the Influence of these a((justmonts on SOA 
actions that affect clients served* services 
provldeOi or service costs 

• whether there havp been any unintended effects 

What has been your experience with requesting a 
special a4)ustment because of unique or unusual 
cl rcumstancos 

What further Improvements could be mado to the 
models 



3. Statj Design of the Incontlvo Rewards Systeo 
How do you understand: 

• the state Intent In designing the r awards 
system 

• what minimal requirements any SOA must meet to 
1 "I 8 qualify for an award (e.g.* certain standards 

that muit bo fnQt# « certiln nunbor that must 
bd met) 
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0. 3. (conttnued) 



SDA 

Administrators Chief Planner, SOA 
(Otrector» Program Oevelop- Intake 
-Deputy 01 rector) mont ^t^ff Director 



Serv tee 

Provider 

Monitors 



Eval uation 
Staff* 

Mrs Staff 



Fiscal 
Officer 



what formula determines the amount of incentive 

rwards that each SOA will get 

the possible strategies for an SOA to maximizo 

or improve its share of the available rewards 

whether the rwards system has influenced SOA 

actions that affect clients served, services 

provided, or service costs 

whether there have been unintended effects 

whether there has been a "learning curve" that 

Will make further effects different from the past 



What has been your pattern of ro^ards. 



4. State Sanctioning Actions 

What has been your experience, if any, in roceivinq 
stata sanctions. 



Kow do you understand: 

• the intention of state sanctions 

• what performance levels have been determined to 
be sancti enable 

• the influence of sanctions or the *iireat of 
sanctions on SOA actions affecting clients 
served* services provided, or service costs 

• whether there are any unintended effects 

• Whether there has been a "learning curve" that 
will make future effects different from the past 



5. SOA Actions Regarding util ization of State Rewards 
for Past Performance 



Horf have past ro^ards been utilized. 

What are plans for utilizing future rewards. 

Is there any effort made to pass rewards along 
to high performing service providers. 

Is there any effort made to use rewards to 
increase services to special need populations 
or fund innovative projects. 
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SOA 

Administrators Chief Plafiner, SOA Service Evaluation 
(Director, Program Oevolop- Intake Provider Staff* Fiscal 
Oeputv Director) ment Staff Director Monitors MIS SUff Officer 

5* (continued) 

Have SDA actions on use of rewpxds had any 
Influence on service provider actions that 
affect clients served, services provided* or 
service costs* 

Has there been a "learning curve" thft will 
make ^uti're effects different from the past. 



6* SDA Actions Regarding Receipt of State 6% Funds Set XX X 

Aside for Hard-to-Serve Clients 

Does the state set aside special funds for 
hard-to-serve clients* 

How does an SDA go about obtaining these funds — 
by performing well on extra state standards* by 
exceeding required levels of service to hard- 
tc-sorve cl lents. 



Old the SOA compete for/ receive any state funds 
set aside for hard-to-sorve clients* Describe 
any protS^cts utilizing these funds* 



Ee The Impact of Other Federal, and/or State Policies 
on SDA Policies and Practices 












1. 


Federal Limitation on Supportive Services Costs 


X 


X 


X 






2. 


Federal Limitation on Administrative Expenditures 


X 


X 


X 






3. 


Federal Requirement to Spend A0% on Youth 


X 


X 








4. 


Federal JTPA Reporting Requirements 














• Related to client-level termination outcomes and 
model factors 

• Related to expenditures 

t Related to post-program outcomes 


X 

X 
X 






X 

X 
X 


X 


5. 


Regulations on Use of 6% Funds 


X 


X 


X 






6. 


Federal JTPA Auditing Practices 


X 








X 
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SDA 

Admtntstrators Chtef Planner, SDA Servtce 

(Ot rector. Program Develop- Intake Provider 

Oeputv Director) ment Staff Director Mr>n<i'or<^ 



Eval uatlon 
Staff, 

MTS <;taff 



Fiscal 
Officer 



E* (continued) 



I 



7. Federal Technical Assistance on (1) the 
Performance Standards; (2) Developing Youth 
Competency Systems; (3) Implementing Inno- 
vative Service Programs 

8. Other Federal Influences such as Federal Welfare 
Reduction Initiatives; Perceived Congressional 
Concerns 

9. Any Additional S '^te Reporti ng Requi rements 

10. SDA Opinion About the Adequacy of JTPA Reporting 
Requi rements 



concerns about data consistency: definitions/ 

data collection practices 

concerns about burden: total cost ^nd staff 

effort 

concerns about the adequacy of measuring model 
adjustment factors 

concerns about the adequacy of follow-up data 
collection practices and/ or data elements 
perceptions about YEC reporting practices 
suggestions for Improving reporting requirements 
or practices 



12. 



13. 



14. 



11. State Role In Designing/Monitoring .MIS: 

• client level termination data and model factors 

• expenditure data, expenditure rates 

• post-program/fol low-up data 

State /tonltorlng Procedures: 

• frequency 

• emphasis on Interim performance 

• scope of state monitoring 

State Goals for Service to Significant Segnents 
or Equitable Service Provision to Dropouts and 
WIN Registrants 

Existence of State Policy to Rescind and 
Reallocate Unexpended SDA Funds 



15. State Policies Regulating Perf onmance-Based 
Contracts for JTPA 



X 
X 
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SOA 

Administrators Chief Planner^ SDA Serv ice Evaluation 

(Director^ Program Develop- Intake Provider Staff* Fiscal 

Deputy Director) ment <;taff Director Monitors MIS Staff Officer 



E* (continued) 



16. State Technical Assistance on? (l) Performance 
Standards; (2) Development of Youth Competency 
Systems; (3) Innovative/Effective Program Design 

17. State Efforts to Coordinate JTPA Title IIA 78% 
Funds with Other JTPA/Non-JTPA Resources 

18. Other State Factors, Such as State Welfare 
Reduction Initiatives* State Legislature Concerns, 
etc. 

For topics 1 through 18 above* the questions Include: 

0 hew SDAs understand the federal or state Intentions 
Implementing these policies or undertaking these 
actions; 

• whether they have Influenced SDA actions that 
affect clients served* services provided* or 
service costs; 

• whether they have had any unintended effects; 

• whether there has been a "learning curve" that 
will make future effects different from the past. 
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OUTLINE OF TOPICS FOR ON-SITE DISCUSSIONS; PIC REPRESENTATIVE 

QU/IITATIVE WflLYSIS 
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OUUINE OF TOPICS FOR ON-SITE DISCUSSIONS: PIC REPRESENTATIVE 

QUALITATIVE ANALYSIS 



1. The Role of the pIC 

What are the primary concerns of PIC members? 

Specifically, what Is their Interest In policies on priority 
clients, service design, contractor performance requi ranents, 
strategies to maximize SDA rewards. 

What Is their Involvement In operational Izing client selection, 
contractor selection, contractor monitoring, assessing SDA per- 
formance, a K 

2. SDA Goals on Clients Served 

Does the SDA have priorities for serving Individuals with parti- 
cular characteristics or members of particular target groups. 

Have priority groups changes from year to year. 

Is there a conscious relationship between the SDA's priority 
Client groups and Its performvHnce objectives. What Is the 
Influence of the state Incentive policy. 

What role does the PIC play In establishing these priorities. 

3. SDA Goals on Service Design 

What Is the SDA's plan for allocating resources across different 
types of services, and how was It determined. 

Are certain types of services emphasized. Are others excluded. 

Is there a conscious relationship betweer, the SDA's program 
design and its performance objectives. What Is the Influence of 
the state Incentive policy. 

What role does the PIC play In the development of the SDA's 
service design. 



4. SDA Development of YEC Systeo 

Does the SDA have YEC system In place. 
What types of YEC categories were used. 
How was the choice of YEC made. 
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(conti nued) 

What was the role of the PIC and other groups. 

What was the role of performance standards in developing YECs. 



SDA Perfonnance Ofcjectives 

What are SDA priorities and objectives regarding: 

• relative Importance of the different standards. 

• the Importance of maximizing financial rewards. 

• desired performance levels on each standard and how arrived 
at (e.g., in comparison to last year, in canparison to this 
year, in comparison to model -acy usted standards, in compari- 
son to the national standard, in canparison to other SDAs). 

How did the local employer needs influence SDA performance 
objectives. 

Do you have local performance goals that are not adequately 
reflected by the federal or state performance standards. 

Does the SDA place different priority on the standards than does 
the state (as reflected in the incentive awards). 



PIC Opinion About Perforaance 

What do you think about SDA performance in PY86. What change 
has there been over time. What changes would you like to see in 
the future. 

What do you think about who was served In PY86. What changes 
would you like to see in who is served. 

What do you think about the range of services provided in your 
SDA in PY86. What changes would you like to see in service 
provision, the types of providers, or provider contractual 
arrangements? 

For all of the above, hew would performance standards constrain 
you in making the desired changes. 



Sunmary Yieiis of Performance Standards 

Overall, hcM useful are the performance standards as measures of 
desi red performance. 

Do the planned post-program standards represent an Improvement 
over the termination-based standards. 
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(continued) 

Do you think that the public, including local elected officials 
and employers, has a different imaqe of JTPA than it had of 
CETA. 

If so, how much do you think the change of image has to do with 
performance standards, and how much with other changes that were 
instituted at the same time (e.g., shift to private sector 
orientation, prohibition of public service employment, limita- 
tion on stipends, etc.). What specific changes in public 
perception have resulted from the performance standards. 
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OUTLINE OF TOPICS FOR ON-SITE DISCUSSIONS: SERVICE PROVIDERS 

QUALITATIVE ANALYSIS 
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OUTLINE OF TOPICS FOR ON-SITE DISCUSSIONS: SERVICE PROVIDERS 

QUALITATIVE ANALYSIS 



A. History of Service Provider as an Emplpynjent Services Contractor 

1. History of Organization as Service Provider for CETA/JTPA 

• Organizational goals and objectives 

— Larger organizational mandate 

• Number of years experience as a provider 
Reason for participating In JTPA as service provider 

• Range of services provided under JTPA 

• Previous experience with special target groups, If any 

• Other funding sources and service contracts 

— How does JTPA fit Into larger organizational goals, 
management practices 

2. Range of Services Currently Provided Under JTPA Contract 

• Number of different service sites 

• Number of distinct contracts with SDA 

• Number of distinct training curricula, services 

• Scale of different service components (enrollment) 

3. Experience Bidding for and Negotiating Service Contract with, SDA 

• *j'^p™ggg^°'"P®^^^^°" ^° y°" tti^nk there was for this contract 

• What is the service provider's perception about what critera 
were used to select contractors: 

— proposed costs 

experience in operating programs 

— specific service design 

— communi-ty orientation 

— "type of provider 

— commitment to hard-to-serve cl ionts 

- • Does the service provider in turn subcontract with any other 
organizations 

— hew are they selected 

— what are their responsibilities/contract payment terms 

B. Contractual Relationships/Relative Roles of SDA &,nd Service Provider 
1. Contract Requirements 

How was the contract negotiated (what actors, how much discus- 
sion, about what points) 
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(conti nued) 

What did PY1986 contracts require regarding: 

a different performance payment points (e.g. enrollment, com- 
pletions, entered employment, tral ni ng-rel ated placements, 
retained employment for "x» period) v,«iiBMi.i„ 

• minimum required performance levels 

• achievement of YECs 

• consequences of failure to meet minimum levels 

• compensation rate for each payment point 

• bonuses for high performance 

• required client characteristics 

• consequences of failure to meet required characteristics 
How did these contractual requirements vary: 

• by type of client 

• by type of service. 

If not performance based, what were reimbursement arrangements. 

Have contractual requirements changed over time for this service 
provider (what were the reasons for changes in contract terms). 

What does provider think about the advantages and/or disadvan- 
tages of performance based contracting. 

r^^?^''''^"^!^ ^^^^""^ provider like to make in contractual 

requiranents and why. 

Description of SDA Monitoring Activities 

Describe SDA monitoring activities: 

• frequency 

• relation to contractual requiranents 

• attention to Interim problems of under-perf ormance 

• requests for corrective action 

Description of SDA Reporting Requirements 

What requirements has the SDA imposed for reporting: 

• applicant characteristics 

• client characteristics 

• attendance 

• termination outcomes 

• follow-up outcomes 

• reporting on YEC outcomes 
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B. (continued) 

4, Provider Opinion about Adequacy of Reporting Requirements 

• concerns about data consistency: definitions and practices 
(e.g., definitions of enrollment, placement, termination) 

• concerns about burden of data collection 

a concerns about the adequacy of adjustments i or client 
characterl sties 

o concerns about the adequacy of follow-up data collection 
practices 

• suggestions for Improving reporting requirements or practices 



C. Service Provider Priorities, Incentives^ and Policies 

1. Factors Influencing Provider Behavior 

What Is the relative Importance to the service provider of: 

• the agency»s own priorities 

• SDA contractual requirements 

• state or federal performance standards facing the SDA. 

Which of the following outcomes ere the most effective deter- 
minants of service provider behavior: 

• Immediate compensation 

• future contracting opportunities 

• community perception of provider performance 

• prov1der»s own priorities, or those of parent organization 

How do these different Incentives differ In how thsy affect 
service provider behcivlor (e.g., program design or client 
selection decisions) 

2. Service Provider Performance Objectives 

What Is the relative Importance to you of the different perfoi^ 
mance requirements In your contract. 

What Is your desired performance on each requirement (e.g. to 
meet the requirement, to exceed the requlr^ent). 

What are the tradeoffs among performance dimensions (I.e. areas 
Where Improving performance on one measure tends to Impede 
performance on another measure)* 
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(contlnuec") 



What Is your strategy for resolving f.rado-offs among performance 
dreas* 

What has your past experience been, in terms of meeting or not 
meeting your performs; ,ce requlreinents^ 

What has been the result of past failures to mc-st performance 
reqolremfenta. 

What has been the result (benefits, rewards) of past experience 
In performing well, conpared to your contract requirements or 
compared to other service providers. 

If you were to have difficul-ty in the future in 'eeting your 
contractual requirements, what 'would your stratyv^y be for 
improving your performance (what effect would it have on v-ho you 
serve, what services you crfer, and the duration or cost of 
sej'vices). 

In PY1986, how well did you do on each of tho performance 
requirements in your concrac-;.. 

What do you think about your PY1986 performarce level. What 
changes would you like to see in your performance in the future. 

What do you think about the rang^ of services you provided in 
PY1986. What changes would yau like to see in vh^' pattern of 
services provided in the tuti; 

What do you think about who you served iu PY1986. What changes 
would you like to see in w^o is seryed in the future. 

For all the above, how would ycur perfcrmance requirements 
constrain you in making the desired changes. 



. Effect on Service Provider of SDA Policies Related to tho 
Perfortnance Standards 

How familiar is the service provider with thj ses' en federal 
perfomrance standards and any additional state standards, and 
with the numerical standards set for the SDA. 

Which performance standards does the SDA think are the most 
important. Coes the service provider agree with this oruer of 
pri orities. 
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3. (continued) 



How does the SDA communicate Its performance priorities to its 
service providers: 



• through general pol icy statements 

• through actual contract requirements 

• through a payment schedule that offers bonuses for higher 
performance levels. 

How do the numerical contract requirements established for each 
service provider relate to the actual numerical performance 
standards set for the SDA as a whole. 

How do the SDA's performance goals, as communicated to Hs 
service providers. Influence service provider practices, such as 
client outreach, client selection, assignment to services, 
design of service content or duration, placement strategies. 

Description of Service Provider Practices 

1. Describe Service Provider Practices Regarding Client Outreach. 

• Is special outreach used to generate applications from 
specific types of applicants (e.g. youth, adults, more quali- 
fied applicants, more difficult to serve clients) 

2. Describe Service Provider Practices Regarding Client Assessment, 
Selection, Enrol Imenii Assignment to Services, Provision of 
Supportive Services 

• What criteria are used to select from among JTPA-el Igibl e 
applicants — are employer Identified (pre-approved) refer- 
rals made for on-the-job placement slots 

© How do the number of eligible applicants compare to the 
available number of training slots. Is a waiting list used. 

• What assessment tests are used by the service provider. How 
are the results of these tests used to determine who is 
admitted; who Is not accepted; what services will be offered 
to an appl leant. 

• What are examples of the reasons for discouraging/excluding 
an applicant frcxi> participating (e.g. basic educational defi- 
ciencies, motivational deficiencies, lack of particula,- apti- 
tudes, too job ready, insufficient supportive services 
available* lack of transportation). 

• When does enrollment occur, and who enrolls clients. 
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2. (continued) 



9 How Is a client assigned to a particular service. How 
frequently are clients enrolled into two services 
simultaneously. How frequently are plans made to enroll 
clients into two services sequentially. 

• What kinds of supportive services are available — 
problems/ issues associated with supportive services/lack of 
sti pends. 

3. Describe Placement* Termination and Follow-up Practices 

« Hew is placement defined* for the purpose of reports to the 
SDA. 

• Does the definition of placement differ by type of service. 

• How is placement defined* for the purpose of payment 
schedules. 

© What is the extent of the responsibility of the service 
provider in the placement process. Are there other service 
contractors who specialize in job placement. 

o What is the extent of service provider responsibility to 
provide follcw-up services to the client (including 30# 60, 
90 day contracts if service related) 

• What is the extent of service provider responsibility to 
generate follow-up data for reporting purposes. 

~ for SDA 

as part formal post-terminati on reporting required by DOL 
a When are clients terminated by the service provider. 
0 When are clients terminated by the SDA. 



Effect on Service Provider of SDA Policies Regarding Client Priori- 
ties and Client Selection 

!• Does the SDA have priorities for serving Individuals with parti- 
cular characteristics or monbers of particular target groups. 

• Does the service provider agree with these priorities. 

d Does the service provider have priority groups of its own 
that differ or supplement the SDA^s priority groups. 

2. How are the SDA client priori 1y groups communicated to service 
providers, and how do they effect the expectations placed on 
particular providers: 
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2. (continued) 



• through general policy statements 

• through contractual requirements 



• through a payment schedule that offers bonuses or a higher 
rate for services to more difficult clients 

• other 

^* yea^.*^Vs°^'ho^'°''^*^ ""^^^"^ groupings changed from year to 

4. How do the SDA's client priorities, as communicated to Its 
service providers. Influence service provider activities, such 
as client outreach, client assessment, client selection, assign- 
ment to service, design of service content and duration, place- 
ment strategies. 

5. What Is the service provider's opinion about who are the hardest 
clients to serve under JTPA. What are the barriers to serving 
them. What are the service provider's strategies. If any for 
serving clients perceived as "hard to serve". 



Effect on Service Providers of SDA Policies Regarding Service Design 
and Assignment to Serv. s Issues ^ a sj" 

1. What Is the SDA's plan for allocating resources across different 
types of services 

2. Does the service provider agree with the SDA's emphases among 
different types of services (e.g., adult versus youth services- 
classroom training versus on-the-job training; in-school youth 
programming versus out of school youth programming; pre- 
employment preparation versus training versus job search and 
placement services). Does the service provider thin.< other 
services are Important that are not emphasized by the SDA. 

3. Hew are the SDA's service priorities communicated to service 
providers, and how do they affect the services offered by speci- 
fic providers: ' 

9 through general policy statements 

0 through specific contract requirements 

• through a payment schedule that offers different rates for 
the provision of different kinds of services. 
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(continued) 

4. How do the SDA»s service priorities, as communicated to its 
service providers, influence service provider activities, such 
as client outreach, client assessment, client selection, 
'-signment to services, design of service content or duration, 
or placement strategies* 

5* Are providers encouraged to/di scourag9d from making service 
referrals to link clients to additional needed services* 

Effect on Service Providers of SDA Policies Regarding Acceptable 
Service Costs 

!• Hew does the SDA specify the acceptable range of service costs 
per enrol lee or per placement 

— in RFPs announcing available funds 

— in contract negotiations 

— in unit price rates 

2* Hew do the acceptable cost ranges vary 

— for different types of services 

— for different -types of clients 

— for different types of providers 

3* Does the service provider think that the available cost 
guidelines set by the SDA are reasonable. What changes would 
the service provider like to see in cost guidelines* Wi^at 
difference would this make in service content or client 
selection practices? 



Issues Relating to Youth Competency Areas (For Providers Serving 
Youth) ^ 

1. What are the YEG outcomes used for your program in particular. 

2. How are th^ applied. 

• What Is the operational definition of a deficiency in each 
area. 

• What Is the operational definition of obtaining a competency 
in each area. 

• Hew do these relate to your payment points? Do you r cord 
YEG outrones for youth In placement oriented programs? 

3. What role did service provider play in developing YEGs In your 
organization. 
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H. (continued) 



4. What are your concerns about the reporting of YEC outcomes, and 
now could this system be improved in your SDA. 

^* pl^Vt^u^ """"^ Factors on Service Provider Practices and 

Size and characteristics of JTPA eligible pool and applicant 



S. 



2. Types of local job opportunities: 

• types of j obs 

• stability of employment 

• wage rates 

3. Geographic size of SDA 

4. Level of SDA's total dollar allocation 



Organized political influence of or on behalf of particular 
constituencies (e.g. various ethnic groups, welfare recipients, 
non-English speaking, disabled). 



Hi 
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